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ABSTRACT 



This report presents an analysis blueprint for data provided 
by the National Education Longitudinal Study of 1988 (NELS:88), a survey 

system sponsored by the National Center for Education Statistics (NCES) . The 
report will inform beginning NELS:88 data users of the many substantive 
issues that can be addressed by the data, and it should present experienced 
researchers with promising ideas that can be explored through NELS:88 data. 
Other audiences may find the report useful in gaining an overview of the 
richness of NCES data in addressing education policy issues. The report, 
primarily the outcome of an NCES -sponsored conference, incorporates the more 
than 70 research issues discussed at the conference, and categorizes them 
into these broad areas: (1) secondary education; (2) transition from school 
to work; (3) postsecondary education; (4) career attainment and life cycle 
transition after college; (5) motivation, aspiration, self- concept, and 
other psychosocial issues; and (6) methodological research. The report is 
organized into four major sections. Section 1 presents information on the 
background and purpose of the report, and section 2 provides an overview of 
the NELS : 8 8 database. Section 3 discusses the types and levels of analysis 
that can be conducted with NELS: 88 data. Section 4 presents a list of issues 
related to the core six areas. Appendixes A-D include technical notes 
(including a glossary) , data analysis suggestions, a discussion of the 
availability of documents and advice, and a listing of other NCES projects. 
Contains one figure and one table.) (SLD) 
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Foreword 



Each year a large number of written documents are generated by NCES staff and 
individuals commissioned by NCES which provide preliminary analyses of survey results and 
address technical, methodological, and evaluation issues. Even though they are not formally 
published, these documents reflect a tremendous amount of unique expertise, knowledge, and 
experience. 

The Working Paper Series was created in order to preserve the information contained 
in these documents and to promote the sharing of valuable work experience and knowledge. 
However, these documents were prepared under different formats and did not undergo vigorous 
NCES publication review and editing prior to their inclusion in the series. Consequently, we 
encourage users of the series to consult the individual authors for citations. 

To receive information about submitting manuscripts or obtaining copies of the series, 
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of Educational Research and Improvement, National Center for Education Statistics, 555 New 
Jersey Ave., N.W., Room 400, Washington, D.C. 20208-5652. 
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I. BACKGROUND AND PURPOSE 



The primary purpose of this report is to present an analysis blueprint for data provided by the 
National Education Longitudinal Study of 1988 (NELS:88), a survey system sponsored by the 
National Center for Education Statistics (NCES), U.S. Department of Education. It is part of a 
continuing effort by NCES to promote and facilitate the use of this valuable national education 
information resource for advancing the understanding of educational and vocational development 
processes. 

This report is designed to serve several audiences. First, it will inform beginning NELS:88 data 
users of the many substantive issues that can be addressed by the data. Second, it will hopefully 
challenge experienced analysts by presenting them with promising but complicated research ideas 
that can be explored with NELS:88 data. Third, other audiences may find this report useful for 
gaining an overview of the richness of NELS:88 data in addressing education policy issues. 

This report is primarily the outcome of an NCES-sponsored conference, held in Washington, DC, 
during the summer of 1995, to discuss a research agenda for the NCES Longitudinal Studies 
Program. Prior to the meeting, NCES staff prepared a general analysis framework which was 
shared with the 40 researchers from a variety of disciplines who were asked to participate in the 
meeting. Invitees were encouraged to review the analysis framework and then prepare a summary 
of research in which they were involved. These summaries were discussed at the meeting and 
served as the primary basis for writing this analysis blueprint. 

Over 70 research issues were discussed at this meeting and incorporated into this report. For the 
convenience of presentation, these research issues have been categorized into six broad areas. 
These areas are: (1) secondary education; (2) transition from high school to work; (3) 

postsecondary education; (4) career attainment and life cycle transition after college; (5) 
motivation, aspiration, self-concept, and other psychosocial issues; and (6) methodological 
research. For each research issue summarized in this report, sections in the specific write-up 
include information on its significance to policy, suggestions for analyses, and cautions on 
potential methodological complications. It should be noted that no attempt was made to prioritize 
the importance of the issues and it is possible that we may have overlooked some issues. 
However, in spite of possible incompleteness, this analysis blueprint should prove to be helpful 
to users in framing their plans to explore the potential of NELS:88 in research and policy 
discussion. 

This report is organized into four major sections. Section I presents information on the 
background and purpose of this report. Section II provides an overview of the NELS:88 database, 
including data file structure and availability of data files to users. Section III discusses the types 
and levels of analyses that can be conducted with NELS:88 and some cautions that users should 
consider in analyzing the data. Section IV presents a list of issues in six broad areas as mentioned 
earlier, including some citations of prior or on-going research. Related technical notes, including 
a NELS:88 glossary, suggestions for analyzing NELS:88 data, the availability of documents and 
technical advice, and a listing of other NCES projects, are presented in appendices A through D. 
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II. OVERVIEW OF NELS:88 DATABASE 



Originally designed as a general purpose data set for the development and examination of 
education policy, NELS:88 can be used to inform decision makers, education practitioners, and 
parents about the changes in the operation of the educational system over time. It can also be 
used to study the relationships of various elements of the system with the educational and 
vocational development of the individuals who go through it. Specifically, NELS:88 focuses on 
a number of inter-related policy issues, including: 

• identification of school attributes associated with achievement; 

• transitions of different types of students from eighth grade to secondary school; 

• transitions of secondary students to postsecondary education or the work force; 

• influence of ability grouping and program type on future educational experiences and 
achievements; 

• determinants of dropping out of the educational system; and 

• changes in educational practices over time. 

NELS:88 also collected information on the role of parents in education. The second followup 
parent survey (the parent survey was also conducted in 1988) can be used to examine the effect 
of parents’ attitudes and behaviors on educational or career choices, financial preparation for 
postsecondary education, the correlates of active parental involvement in the school, and the 
parents’ role in the educational success of their children. 

NELS:88 is the successor of two prior studies sponsored by NCES: the National Longitudinal 
Study of the High School Class of 1972 (NLS-72) and High School and Beyond (HS&B). It 
expands the knowledge base developed by the previous studies by following students from an 
earlier age (8th grade) and by updating information throughout the 1990s. Its major features 
include: 

• the integration of data from students, dropouts, parents, teachers, and schools; 

• the inclusion of supplementary components to support analyses of geographically and 
demographically distinct subgroups; 

• the design linkages to previous longitudinal studies and other current cross-sectional 
studies; and 

• the use of proficiency tests that are "adaptive" to the ability level of the student. 
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NELS:88 began with a cohort of 25,000 eighth graders attending 1,000 public and private schools 
across the nation in 1988. Since then three followup surveys have been completed at two-year 
intervals — 1990, 1992, and 1994. The next followup is tentatively scheduled for 1998 when most 
respondents will have finished postsecondary education and/or have entered the labor force. 

For most survey components, data were collected through self-administered questionnaires/ tests 
in group sessions on campus, but there were several exceptions. 1) Parents were surveyed via 
mailed questionnaires, and 2) dropouts and students who had transferred schools and who missed 
in-school sessions were surveyed through off-campus group sessions. Finally, in the third 
followup, as most cohort members had graduated from high school, computer-assisted telephone 
interviews were conducted to collect data. 

The following subsections summarize the design and content of each NELS:88 survey. For 
further details, readers should consult the Users’ Manuals and other technical documents listed 
in appendix C. 

A. Design and Content of NELS:88 

1. Base-year survey 

The base-year survey was comprised of four components: (a) survey and tests of students, (b) 
survey of parents, (c) survey of teachers, and (d) survey of school administrators. The student 
component provided data on the students’ backgrounds and on such topics as school work, 
educational and occupational aspirations, and social relationships. The cognitive tests completed 
by students were curriculum sensitive, measuring achievement in reading, mathematics, science, 
and social studies. The parent survey, administered to one parent of each sampled student, 
collected data on parental aspirations for children, the education support at home, and family 
characteristics in general. Selected teachers in two of the four subject areas completed the teacher 
questionnaire, providing data on school and teacher characteristics, evaluations of the sampled 
students, course content, and classroom teaching practices. Finally, school principals of the 
sampled schools were surveyed, providing data about each school’s teaching staff, school climate, 
characteristics of the student body, and school policies and program offerings. 

A two-stage stratified probability design was used to select a nationally representative sample of 
8th grade schools and students. Schools constituted the primary sampling unit. An initial pool of 
1 ,032 schools was selected through stratified sampling with probability of selection proportional 
to eighth grade size and with over sampling of private schools. After replacing ineligible and 
nonparticipant schools, a total of 1,052 schools (815 public schools and 237 private schools) 
participated in the study and provided usable student data. The second stage of sampling — student 
sampling — produced a random selection of 26,432 eighth graders from participating schools, of 
which 24,599 participated in the study during the 1988 spring term. On average, each school was 
represented by 23 participating students. Students were randomly sampled from 8th grade school 
rosters. 
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2. First followup survey 

The first followup of NELS:88 was comprised of surveys and tests of students, surveys of 
teachers and school administrators, and three new components — the dropout study, the base-year 
ineligible study, and the school effectiveness study. Students, augmented by a freshened sample, 
completed a questionnaire and a series of cognitive tests. The survey gathered information on 
students’ school and home environment, coursework, extra-curricular activities, employment, 
education aspirations, and self-concept. The test was designed to measure tenth grade achievement 
and cognitive growth in the period in the same four subject areas as in the base year. The teacher 
and administrator surveys were administered in the same way as in the base year. A new student 
supplement was completed by first-time participants, including students who were added to the 
sample through the freshening process, students who were ineligible in the base year but became 
eligible now, and base-year nonrespondents who participated in the first followup. The dropout 
study surveyed and tested youths who had dropped out of school between the spring term of the 
1987-88 school year and the spring term of the 1989-90 school year. 

Students were selected in two stages. The first stage involved the selection of 21,474 students 
who were in the eighth grade sample in 1988. Since some tenth graders in 1990 were not in the 
eighth grade or not in the country in the 1988 spring term, a freshening process was used to add 
1,043 tenth graders into the sample such that the resulting sample represented the 1990 national 
population of tenth graders. The base- year ineligible study was added to the first followup to 
ascertain the 1990 school enrollment status and the 1990 NELS:88 eligibility status of students 
who were excluded from the base-year survey due to language barrier or disabilities. 

Because 90 percent of NELS:88 base year students changed schools between 8th and 10th grades, 
the first follow-up school sample was no longer nationally representative. To sustain analyses of 
high school issues with longitudinal data, it was necessary to conduct a school effectiveness study 
in conjunction with the first followup survey. This study involved a sizable and representative 
within-school student sample that consisted of 248 NELS:88 schools from 30 major metropolitan 
areas. As participants of the first followup, these schools cooperated in additional data collection 
for the school effectiveness study. 

3. Second followup survey 

The second followup repeated all components of the first followup, re-included the parent 
component, and initiated two new components — the transcript and course offerings components. 
The course offerings component was implemented as a part of the school effectiveness study. The 
transcript component was undertaken for sample members who were students attending sampled 
schools in the spring of 1992, all dropouts, dropouts in alternative programs, early graduates, and 
"triple ineligibles" (those 1988 eighth graders who were ineligible for the base year, first 
followup, and second followup due to language barriers or disabilities). 

The student component again consisted of a questionnaire and a series of cognitive tests. The 
questionnaire focused on planning for postsecondary education and work, among other topics. The 
test measured twelfth grade achievement and cognitive growth in the same four subject areas. The 
student questionnaire contained a supplement for early graduates, which tapped the reasons for 
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and circumstances of early graduation. From first-time participants in NELS:88 (either via the 
freshening process or the ineligible study), another supplement questionnaire collected 
demographic background data. While teachers and administrators were surveyed again, only one 
teacher for each student — rather than two as in previous waves — was asked to participate. 

Each student and dropout selected for the first followup was included in the second followup. 
From the schools attended by the sample members, 1,500 schools with 12th grade were included 
into the school sample. All components were completed in 1,374 schools, but only the student 
and parent questionnaires were administered in the remaining 126. The student sample was then 
freshened to provide a representative sample of students enrolled in the 12th grade during the 
spring term of the 1991-92 school year. 

The second followup school effectiveness study included 247 participating schools. The sampling 
process involved freshening and additional selection of students. Data were enhanced by the 
course offerings component and the free response science and mathematics test items given to this 
sample. 



4. Third followup survey 

In 1994, the third followup interviewed students via telephone after they were out of high school. 
The student component covered information on high school completion and alternative ways of 
completion, post high school employment, schooling, and family life. The sample was selected 
based on the retained sample of the second followup, with subsampling based on relevant policy 
subgroups and verified eligibility. The sampling design involved stratification on a number of 
variables, including the original high school sector, race, SES quartile, and high school dropout 
status. 

B. Alternative Student Transition Paths 

To understand the complicated survey design and file organization, data users need to understand 
the transition paths that a student may have followed between the 1988 base-year and the 1992 
second followup surveys. For example, the status of a student surveyed as an eighth grader could 
change several times between the base year and second followup (e.g., student in 1988, dropout 
in 1990, and student in 1992). By the time this student was surveyed in 1994, he/she might have 
completed high school, earned a GED, or still not have a high school diploma. 

The above example is further elaborated in figure 1 which presents the statuses (and numbers) 
of NELS:88 sample students who followed selected paths as of the second followup. For 
example, out of 20,062 students (column 1) who were sampled in the base-year and retained in 
the first followup surveys, 18,270 (column 2) were classified as students in the first followup. 
Of these 18,270 students, 16,885 (column 3 — 69 + 16,339 + 11 + 466) were classified as students 
during the second followup. Researchers are advised to pay particular attention to the shifts 
between dropouts, students, out of scope, status unknown respondents from the first to the second 
followup surveys. 
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Figure l.~NELS:88 eighth-grade spring defined cohort status distribution in first and 

second follow-ups 
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C. Available Data Sets 

Given the number and breadth of coverage of the individual NELS:88 base-year through the third 
followup surveys, researchers have much latitude in selecting sub-samples and variables for 
specific analyses. In fact, with the completion of the third followup, NELS:88 data can be 
categorized into the following data sets: 

1. Four waves of student data (1988, 1990, 1992, and 1994) 

2. Three waves of dropout data (1990, 1992, and 1994) 

3. Three waves of middle/high school data (1988, 1990, and 1992) 

4. Three waves of teacher data (1988, 1990, and 1992) 

5. Two waves of parent data (1988 and 1992) 

6. Three waves of cognitive test data (1988, 1990, and 1992) 

7. Two waves of freshened student data (1990 and 1992) 

8. Base-year ineligible students (added in 1990 and 1992) 

9. Asian and Hispanic student supplement (augmented in 1988; followed in 1990 and 
1992) 

10. Math and science item supplements (1988, 1990, and 1992)— additional items 
sponsored by the National Science Foundation 

11. School effects supplement (1990 and 1992) — restricted-use data* 

12. School effects supplement math and science free response item file 
( 1 992) — restricted-use 

13. School effects supplement school course file (1992) — restricted-use 

14. High school transcripts (1992) — restricted-use 

15. Cognitive test item response level file (1988, 1990, and 1992) — restricted-use 

16. Base-year school organizations supplement (1988) — restricted-use-supplemental 
items asked in a re-survey of base year schools 

17. Christian school supplement (1988 and 1992) — restricted-use 

18. State augmentations for selected states (1988, 1990, 1992, and 1994) — restricted- 
use 

19. Early graduate supplement (1992) 

20. Census zip code data file (1992) — restricted-use 

21. Common Core of Data merge IDs (1992) — restricted-use 

22. Quality education data selected school variables (1992) — restricted-use 

* NCES releases the NELS:88 data files on CD-ROMs in two different formats: public 
release and restricted-use release. Access to the restricted-use CD-ROM requires a special 
licensing agreement with NCES. These can be obtained by calling the Data Security 
Officer at (202) 219-1920. 

D. NELS:88 Data File Structure 

NELS:88 data, both public and restricted-use, are available on CD-ROMs. Each CD-ROM 
contains (1) data directories, (2) document directories, (3) readme files, and (4) electronic 
codebook directories containing software for viewing individual survey questions and frequencies. 
Both the public and restricted-use third followup CD-ROM contains two electronic codebooks 
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(ECB). The first ECB contains data for the base year through second followup (every case who 
has ever been in the sample) and the second ECB contains data for respondents who only 
participated in the third followup (base year through third followup data). Each of the electronic 
codebooks allows the user to: 

1. Display unweighted frequencies and percents for each variable; 

2. Display question text for each variable; 

3. Select or tag variables for subsequent analyses; 

4. Write SAS, SPSS/PC+, and SPSS for Windows program codes and command 
statements for constructing a raw data file or system file of the selected variables; 
and 

5. Generate a printed codebook of the selected variables. 

To operate the NELS:88 ECB, it is recommended that the user have access to a PC-DOS based 
computer (486 or higher) with a CD-ROM drive, a 1 00 megabyte hard drive and 4 megabytes 
of ROM (at a minimum), and SAS/PC or SPSS statistical software. 
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ANALYSIS PERSPECTIVES 



A. Type of Analysis 

The NELS:88 design enables researchers to conduct three types of analyses: cross-wave 
(longitudinal), cross-sectional at a single time point, and cross-cohort by comparing NELS:88 
findings to those of HS&B and NLS-72. 

1. Longitudinal analysis 

The first of these three types of analysis serves the primary objective of longitudinal surveys. The 
NELS:88 sampling and data collection design gives priority to maintaining and surveying a 
substantial number of base-year sample members as well as to sustaining overlapping but 
analytically distinct cohorts of sophomores and seniors. Users of NELS:88 data will be able to 
study the effect of a wide variety of factors on students’ educational and professional attainment. 
The longitudinal data gathered from students, and augmented through parent, teacher, school 
administrator, and school record of students’ progression and development (for example, academic 
transcripts), will facilitate scrutiny of various facets of students’ lives and permit examination of 
the relationships of these factors on social, behavioral, educational, and vocational development. 

2. Cross-sectional analysis 

The second analytic type within NELS:88 is cross-sectional. By beginning with a cross-section 
of 1988 eighth graders, following a substantial subsample of these students at two-year intervals, 
and freshening the 1990 and 1992 samples to obtain representative national cross-sections of tenth 
and twelfth graders, the study also provides a statistical profile of America’s eighth graders, high 
school sophomores, and high school seniors. 

3. Cross-cohort comparison 

Finally, NELS:88 has been designed to provide researchers with data for drawing comparisons 
with previous NELS-sponsored longitudinal studies. To facilitate cross-cohort comparisons, many 
of the content areas and related questionnaire items contained in the NLS-72 and HS&B surveys 
were repeated in NELS:88 surveys. The comparability and consistency of items across these three 
data sets allow us to conduct, among others, the following trend analyses: 

• NELS:88 1990 sophomores (with freshening) can be compared to HS&B 1980 
sophomores; 

• NELS:88 1990 (with freshening) sophomores two years later (that is, in 1992) can 
be compared to HS&B 1980 sophomores two years later (in 1982); 

• NELS:88 1990 (with freshening) sophomore cohort dropouts (as of 1992) can be 
compared to HS&B 1980 sophomore cohort dropouts (as of 1982); 

• NELS.88 1992 seniors (with freshening) can be compared to HS&B 1980 seniors 
and NLS-72 1972 seniors; and 
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• NELS:88 1992 collected high school transcripts can be compared to HS&B 1982 
collected high school transcripts. 

Comparisons are also possible using transcript data collected for 1987 and 1990 high school 
graduates in NAEP schools. Table 1 summarizes the types of trend comparisons possible. 
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Table 1. — Trend comparison points for students, dropouts, parents of seniors, or transcript 
comparisons 

National Education Longitudinal Studies Program 



Students 


NLS-72 


HS&B-So 


HS&B-Sr 


NELS:88 


Grade 










G8 








1988* 


G10 




1980* 




1990* 


G12 


1972* 


1982 


1980* 


1992* 


G12 + 1 


1973 








G12 + 2 


1974 


1984 


1982 


1994 


G12 + 4 


1976 


1986 


1984 




G12 + 5 








1998 


G12 + 6 






1986 




G12 + 7 


1979 








G12 + 10 




1992 






G12 + 14 


1986 








Dropouts 










G10 - G12 




1982* 




1992* 


followup 




1984 

(1986, 1992) 




1994 

(1998) 


Early graduates 


1982 






1992 


Parents of seniors 






1980 


1992 


High School 
Transcript Studies 


HS&B 


NAEP’87 


NAEP’90 


NELS:88 


Seniors in: 


1982 


1987*' 


1990* 


1992* 


Note: Comparison points are in bold. 
Representative grade samples are * (asterisked). 
So = Sophomore; Sr = Senior. 







1 Based on the populations of students in eleventh grade and/or age seventeen in 1985-86. 
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B. Level of Analysis 

Both descriptive and relational analyses can be conducted with NELS.88 data. Descriptive 
analyses, often used to summarize education conditions, can be used to produce a snap shot of 
statistics for one year or statistics across years. For example, achievement test results can be 
generated for a given year or for cross years. They can also be presented in comparison to earlier 
cohorts at the same grade level; for example, achievement by tenth graders from the class of 1990 
in NELS:88 first followup and tenth graders from the class of 1980 in HS&B. Such analyses may 
suggest trends of achievement growth across cohorts. 

Relational analyses usually seek to identify the relationships between an outcome variable and 
the variables that theoretically influence the outcome. This can be done in either cross-sectional 
or longitudinal designs. Results from cross-sectional analyses can be generalized to the national 
population because the NELS:88 student sample was freshened in each wave to represent the 
national population at a given grade level. The longitudinal design, in which the main sample of 
the cohort was followed throughout the first four waves of the survey, allows researchers to 
ascertain relationships between antecedent variables (e.g., demographic background and 
coursework) and outcome variables (e.g., high school completion and college attendance). 
Furthermore, cross-cohort analyses make it possible to see how the relationships revealed by 
relational analyses based on either cross-sectional or longitudinal designs changed in the past 
decades. For instance, with data from NLS-72, HS&B, and NELS:88, we can examine how the 
relationship between high school course taking and college entrance changes among high school 
graduates of 1972, 1980, and 1992. 

C. Unit of Analysis 

By design, the basic unit of analysis for most NELS:88 analyses is the student. The school 
administrator, parent, and teacher data can be thought of as providing contextual data (e.g., 
background, school characteristics, opportunity to learn). The exception to this rule is the base- 
year school file. Because the base year of NELS:88 involved the random selection of 1,000 
schools from across the nation, the eighth-grade school sample can also be considered to be a 
stand-alone file in that the basic unit of analysis can be the school. This is not true of the first 
and second followup school files, however. These schools were selected because they contained 
students who participated in NELS:88 during the base year. Using common IDs, the individual 
data files comprising NELS:88 can be merged with each other to form working data files 
containing student, parent, school, and teacher data. 

D. Sampling Weights 

Because students were not selected with equal probabilities, sampling weights need to be used 
in analyses in order to obtain unbiased estimates. Sampling weights are the inverse of selection 
probabilities adjusted for non-responses. Details about proper weights and correct use of these 
weights are presented in Appendix B. 
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E. Cautions and Limitations 

There are some limitations. For example, the sample sizes for specific groups of respondents may 
be too small to get reliable estimates (e.g., the number of American Indian/Alaskan Native youth 
broken down by gender and socio-economic status). In other instances, NELS.88 may not have 
collected enough information to examine an issue in detail (e.g., drug use history or courses taken 
in elementary school). There are also some nuances in the data such as the high numbers of 
missing data for dropouts who did not take the cognitive tests or who took the abbreviated 
(adapted for phone) form of the dropout questionnaire. Data users are thus advised to study the 
user manuals closely and to conduct preliminary analyses (e.g., frequencies, missing value checks, 
bias analyses) so that the disadvantages and limitations of the data are known prior to the actual 
data analysis. 
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The research issues that can be addressed with NELS:88 data are grouped into six broad 
substantive areas: (a) secondary education; (b) transition from high school to work; (c) 

postsecondary education; (d) career attainment and life cycle transition after college; (e) 
motivation, aspiration, self-concept, and other psychosocial issues; and (f) methodological 
research. Within each area, specific research issues as identified by NCES staff and the 
conference participants are listed. For each issue, a short summary is presented, providing 
information on 1) the significance of the issue to policy making, 2) addressability using NELS:88 
data, and 3) possible cautions due to methodological complications. 

The amount of detail provided in each summary varies. For some areas where much research has 
already been conducted, the summary statements tend to be longer and more detailed. In other 
areas, less research had been conducted and thus the summary may point to the need for future 
research. In these areas, ideas rather than results of past research are presented. 

A listing of relevant literature is included for most research issues. The citations cover only 
research that used NELS:88 data and that were published into 1995. Note that NELS:88-based 
research regarding postsecondary education and later transition is scant in the literature. This is 
because NELS:88 data on postsecondary education, collected in the third followup, are being 
processed and are not yet publicly available. Readers should consult other sources of literature 
to develop theoretic and technical understanding of those research issues. 

A. Secondary Education 

NELS:88 was designed to study the transitions students experienced as they moved from eighth 
grade to high school. It offers great potential for research on secondary education because it 
collected information on academic achievement at three points in time from multiple sources 
(teachers, students, administrators, parents, and transcripts). Important research issues on 
secondary education can be addressed with NELS:88 data. 



1. Linking Instructional Environment/Processes to Learning 

Issue: Achievement growth (categorized by proficiency levels) as related to 
instruction/learning processes for specific student groups 

One of the original purposes of NELS:88 is to trace achievement growth over the high school 
years. NELS.88 has improved its tests, particularly the mathematics test, by taking an adaptive 
test approach whereby both mainstream and disadvantaged students have opportunity to show 
their cognitive growth. Their test gains, however, tend to be in different areas: high ability 
students may gain in problem solving skills whereas disadvantaged students may gain in repertory 
skills. Because their gains may look similar on the aggregated scale, it is necessary to differentiate 
gains at different proficiency levels in performing analyses. To identify the process in which the 
achievement gains take place, one can use transcripts to examine the courses taken by students 




14 N ELS: 88 Research Framework and Issues 



25 



Secondary Education 



who may be at different proficiency levels. The process variables can then be linked to the test 
gains at the corresponding proficiency level to assess the effectiveness of the process. Only after 
separating the proficiency levels in the test score gains, can we both detect the gains and 
understand the meaning of the gains. To tackle diverse learning processes and to check their 
results, using a differential adaptive approach is imperative. 

Relevant literature 



Epstein, J.L. & Maclver, D.J. (1992). Opportunities to learn. Effects on eighth graders of 
curriculum offerings and instructional approaches . Baltimore, MD: Center for Research on 
Effective Schooling for Disadvantaged Students 

Horn, L. & Hafner, A. (1992). A Profile of American eighth-grade mathematics and 
science instruction. NELS:88. Statistical analysis report . NCES 92-486. Washington, DC: NCES. 

Ingels, S.J. et al. (1994). A profile of the American high school sophomore in 1990 . 
National Education Longitudinal Study of 1988 . NCES 95-086. Chicago, IL: National Opinion 
Research Center. 

Jaskir, J. (1994). The Application of proficiency scaling on the measurement of change . 
Unpublished doctoral dissertation. Rutgers University. 

Kupermintz, H., Ennis, M.M., Hamilton, L.S., Talbert, J.E. & Snow, R.E. (1994). 
Enhancing the validity and usefulness of large-scale educational assessment I: NELS:88 
mathematics achievement . Stanford, CA: Center for Research on the Context of Secondary 
Teaching, Stanford University. 

Hamilton, L.S., Nussbaum, M., Kupermintz, H., Kerkhoven, J.I.M. & Snow, R.E. (1994). 
Enhancing the validity and usefulness of large-scale educational assessments II: NELS.88 science 
achievement . Stanford, CA: Center for Research on the Context of Secondary Teaching, Stanford 
University. 

Rock, D., Owings, J. & Lee, R. (1994). Changes in math proficiency between 8th and 
tenth grades. Statistics in brief . NCES 93-455. Washington, DC: NCES. 

Rock, D.A. & Pollack, J.M. (1991). Psychometric report for the NELS:88 Base Year Test 
Battery. Contractor report. Technical report . NCES 91-468. Washington, DC: NCES. 

Rock, D.A., Pollack, J.M. & Hafner, A. (1991). The tested achievement of the National 
Education Longitudinal Study of 1988 eighth grade class. E.D. Tabs . NCES 91-460. Washington, 
DC: NCES. 



Issue: The effectiveness of curriculum and pedagogy in improving student learning 

With its multiple sources of data (students, teachers, parents, and schools) and longitudinal 
design, NELS:88 allows comprehensive research on instruction effectiveness and academic press 
in connection to student performance. Linking teacher reported curriculum data to transcripts 
seems to be a possible way to address the issues. Student reported classroom experience is 
another source to link to the outcomes. 

Caution: The data on instruction in NELS:88 can be problematic. They reflect instruction during 
the short period of time when the survey was conducted; i.e., a year or maybe only a semester, 
rather than the whole two years between waves. It is questionable to relate the short-term 
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instruction features to test scores that reflect the outcome of two years. In addition, measurement 
of instruction tends to be overly simplistic. For instance, in science instruction, the more time 
spent in classroom discussion, the less time spent in hands-on experiments, but both are crucial 
for effective science learning. If we only look at one activity’s correlation with the outcome, the 
complex process of instruction and learning can be misleadingly simplified. The instruction data 
are promising, but need to be handled cautiously in analysis. 

Relevant literature 



Bobbitt, S.A. (1993). Using opportunity to learn items in elementary and secondary 
national surveys . Paper presented at the meeting of the American Educational Research 
Association, Atlanta, GA, April 1993. Education Resource and Information Center Document No. 
ED 363667. 

Braddock, J.H.II & Dawkins, M.P. (1993). Ability grouping, aspirations, and attainments: 
Evidence from the National Educational Longitudinal Study of 1988. Journal of Negro Education, 
62(3), 324-36. 

Braddock, J.H.II & Slavin, R.E. (1992). Why ability grouping must end: Achieving 
excellence and equity in American education . Paper presented at the Common Destiny 
Conference, Johns Hopkins University, Baltimore, MD, September 1992. Education Resource and 
Information Center Document No. ED 355296. 

Horn, L. & Hafner, A. (1992). A Profile of American eighth-grade mathematics and 
science instruction. NELS:88. Statistical analysis report . NCES 92-486. Washington, DC: NCES. 

Rasinski, K.A. & West, J. (1990). Eighth graders’ reports of courses taken during the 
1988 academic year by selected student characteristics . NCES 90-459. Washington, DC: NCES. 

Thomas, J. (1990). The Numbers game: What eighth graders study — and how much of 
it they do. Humanities, 1H 6). 9-10. 



Issue: Opportunity to learn (OTL) with alternative definitions to distribution of OTL 
across student groups 

Alternative definitions for OTL may use a variety of variables including content of instruction, 
instructional strategies, resources and expenditures, availability of technology, staff quality, and 
compensation. These alternative definitions can be used to examine the differential distribution 
of OTL by student needs. A question that can be asked is how tangible resources (e.g., money 
and technology) lead to shaping a positive learning environment, which further leads to higher 
achievement. The results from such an examination can help policymakers and the administration 
monitor the condition of OTL. 

NELS:88 contains a large number of data items to tap OTL. Common Core of Data (CCD) and 
other sources of data can be merged into NELS.88 data (with the common identification number) 
to expand the concept of OTL. Scale construction may be needed for developing alternative OTL 
definitions. Descriptive analysis of OTL by student groups (e.g., sex, race, SES, locale) can help 
identify inequality. Structural equation and HLM techniques may be used to link OTL to 
variables responsible for differential OTL. 
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Caution: Instruction data represent the processes in the short period of time when the survey was 
conducted and are thus inadequate to explain the outcomes of the two years of schooling between 
the followups. 

Relevant literature 



Bobbitt, S.A. (1993). Using opportunity to learn items in elementary and secondary 
national surveys . Paper presented at the meeting of the American Educational Research 
Association, Atlanta, April 1993. Education Resource and Information Center Document No. ED 
363667. 

Braddock, J.H.II & Dawkins, M.P. (1993). Ability grouping, aspirations, and attainments: 
Evidence from the National Educational Longitudinal Study of 1988. Journal of Negro Education, 
62(3), 324-36. 

Burks, L.C. (1994). Ability group level and achievement. School Community Journal,#4 
(1), 11-24. 

Braddock, J.H.II & Slavin, R.E. (1992). Why ability grouping must end: Achieving 
excellence and equity in American education . Paper presented at the Common Destiny 
Conference, Johns Hopkins University, Baltimore, MD, September 1992. Education Resource and 
Information Center Document No. ED 355296. 

Catsambis, S. (1994). The path to math: Gender and racial-ethnic differences in 
mathematics participation from middle school to high school. Sociology of Education, 67 (3), 199- 
215. 

Catsambis, S., Mulkey, L., Crain, R. & Ascevedo, M. (1994). Gender and middle-school 
tracking effects on student attitudes, engagement, aspirations and locus of control . Paper presented 
at the meetings of the American Sociological Association, Los Angeles, CA, August 1994. 

Epstein, J.L. & Maclver, D.J. (1992). Opportunities to learn. Effects on eighth graders of 
curriculum offerings and instructional approaches . Baltimore, MD: Center for Research on 
Effective Schooling for Disadvantaged Students. 

Horn, L. & Hafner, A. (1992). A Profile of American eighth-grade mathematics and 
science instruction. NELS:88. Statistical analysis report . NCES 92-486. Washington, DC: NCES. 

Stevenson, D.L., Schiller, K.S. & Schneider, B. (1994). Sequences of opportunities for 
learning. Sociology of Education. 67 (3). 184-98. 



Issue: Academic growth profiles to related background and curriculum factors 

Study of academic growth conceptualized as group profiles can typify growth rates and the shape 
of the growth curve differentiated by important school and sociodemographic factors. More 
responsive instruction can be developed according to meaningful growth profiling. Particularly 
interesting will be growth profiling at high proficiency levels, as the results will reflect the 
learning of problem solving skills (one of the Goals 2000). 

With NELS:88 test score gains, three growth profiles can be generated: (1) "early accelerated 
growth," which includes students who made their biggest gain between grades 8 and 10, (2) "later 
accelerated growth," which refers to those who gained more between grades 10 and 12, and (3) 
"constant growth," which covers those who made about the same gains across the four year 
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period. The three profiles can then be correlated to background (e.g., SES, sex, and locale) ahd 
schooling (course work, curriculum, and school policies). Data on level 4 in math and level 3 in 
science may generate high proficiency profiles for assessing the gap from the Goals 2000. 

Relevant literature 



Jaskir, J. (1994). The Application of proficiency scaling on the measurement of change . 
Unpublished doctoral dissertation. Rutgers University. 

Catsambis, S. (1994). The path to math: Gender and racial-ethnic differences in 
mathematics participation from middle school to high school. Sociology of Education, 67 (3), 
199-215. 



Issue: Teacher characteristics, teacher-student interaction, and school performance 

Study is needed to focus on personal interaction and relationships between teachers and students 
in elementary and secondary schools. Instruction and learning can be studied by examining the 
teacher- student relationship. Teachers’ evaluations and expectations regarding student efforts and 
performance on one hand, and students’ perceptions about classroom instruction and teacher 
attitudes on the other, are jointly responsible for the outcomes. Research on the issue is important 
for pedagogy improvement. 

A unique strength of NELS:88 data is the link between data about student learning collected from 
teachers and data about instruction collected from students. This feature allows researchers to 
delve into the processes in which teaching influences students’ perception on instruction and 
learning, and such perceptions in turn affect performance. 

Caution: Teachers’ responses to pedagogical questions probably cover a period (usually a 

semester or one year) shorter than the two-year time span in which the followup took place. Thus, 
the information provided by the teacher may not fully reflect the pedagogical effects on the 
student outcomes in the two years. 

Relevant literature 



Chaney, B. (1994). Teachers’ academic backgrounds and student outcomes in science and 
mathematics: NSF/NELS:88 teacher transcript analysis . Rockville, MD: Westat, Inc. 

Choy, S.P. & Bobbitt, S.A. (1993). America’s teachers: Profile of a profession . 
Washington, DC: NCES. 

Ehrenberg, R.G. et al. (1995). Do teachers’ race, gender, and ethnicity matter? Evidence 
from the National Educational Longitudinal Study of 1988. Industrial and Labor Relations 
Review. 48 (3), 547-61. 

Talbert, J.E. & McLaughlin, M.W. (1992). Understanding teaching in context . Stanford, 
CA: Center for Research on the Context of Secondary Teaching, Stanford University. 

Taylor, D.L. & Tashakkori, A. (1994). Predicting teachers’ sense of efficacy and job 
satisfaction using school climate and participatory decision making . Paper presented at the 
meeting of the Southwest Educational Research Association, San Antonio, TX, January 1994. 
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2. Equity and Access 

Issue: The performance of the current school system in reaching the goals of equity and 

access 

Special populations in NELS:88 include limited English proficient (LEP) students, special 
education students, schools in disadvantaged locales (inner city or remote rural places), low SES, 
and racial/ethnic minorities. These groups have different needs and learning approaches, and face 
different school environments (school resources and climate) and variant opportunities to learn 
(e.g., course content, amount, and instruction strategies). 

Two student populations in particular concern researchers. One is students in urban areas, 
surrounded by violence and poverty; the other is Hispanics with extremely low literacy. The two 
populations are growing rapidly and pose serious policy questions relating to their education. 

NELS:88 offers information for studying the two groups, with oversampling of Hispanics. Many 
pressing issues related to the two groups can be addressed. For instance, to what extent do the 
two groups respond to program conditions? Are there interaction effects between at-risk factors 
and program conditions? In other words, does a specific program work with a given risk group 
more effectively than it does with another group? 

Caution: Urban schools and other disadvantaged, "outlier" schools may be under-represented in 
NELS:88; the existing definition of urban schools is questioned by some researchers as too 
narrow. 

Relevant literature 



Anderson, J. et al. (1992). Poverty and achievement: Re-examining the relationship 
between school poverty and student achievement . Paper presented at the meeting of the American 
Education Research Association, San Francisco, April 1992. Education Resource and Information 
Center Document No. ED 346207. 

Bradby, D. (1992). Language characteristics and academic achievement: A Look at Asian 
and Hispanic eighth graders in NELS:88. Statistical analysis report . NCES 92-479. Washington 
DC: NCES. 

Braddock, J.H.II & Dawkins, M.P. (1993). Ability grouping, aspirations, and attainments: 
evidence from the National Educational Longitudinal Study of 1988. Journal of Negro Education. 
62(3), 324-36. 

Braddock, J.H.II & Slavin, R.E. (1992). Why ability grouping must end: Achieving 
excellence and equity in American education . Paper presented at the Common Destiny 
Conference, Johns Hopkins University, Baltimore, MD, September 1992. Education Resource and 
Information Center Document No. ED 355296. 

Burbridge, L.C. (1991). The Interaction of race, gender, and socioeconomic status in 
education outcomes. Center for Research on Women Working Paper Series. No. 246 . Paper 
presented at the annual meeting of the American Sociological Association, Cincinnati, August 
1991. Education Resource and Information Center Document No. ED 360243. 
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Burks, L.C. (1994). Ability group level and achievement. School Community Journal. 
4(1), 11-24. 

Catsambis, S. (1994). The path to math: Gender and racial-ethnic differences in 
mathematics participation from middle school to high school. Sociology of Education, 67 (31. 
199-215. 

Collins, M.A. (1992). School social context, self-esteem, and locus-of-control among 
white, black, and Hispanic youth . Unpublished doctoral dissertation. University of Maryland at 
College Park. 

Hernandez-Gantes, V.M. (1993). Direct and indirect influences of school learning variables 
on Hispanic American eighth-grade students’ academic achievement . Unpublished doctoral 
dissertation. Virginia Polytechnic Institute and State University. 

Lee, V.E. & Smith, J.B. (1992). Effects of school restructuring on the achievement and 
engagement of middle-grade students. Sociology of Education. 66 (31. 164-187. 

Maclver, D.J. (1991). Helping students who fall behind: remedial activities in the middle 
grades . Baltimore, MD: Center for Research on Effective Schooling for Disadvantaged Students. 

Office of Education Research and Improvement. (1994). What do student grades mean? 
Differences across schools . Washington, DC: Author. 

Schneider, B. & Shouse, R. (1992). Children of color in independent schools: An analysis 
of the eighth-grade cohort from the National Education Longitudinal Study of 1988. Journal of 
Negro Education. 61 (21. 223-234. 

Sebring, P.A. & Camburn, E.M. (1992). A Profile of eighth graders in Catholic schools, 
based on the National Education Longitudinal Study of 1988 . Washington, DC: National Catholic 
Educational Association. 



Issue: Comparisons across state and localities in education resources and outcomes 

Since state and local governments take chief responsibility for educational policymaking, resource 
allocation, and program planning, it is particularly appropriate to examine equity of education 
input and outcomes at the state level. 

Using the NELS:88 State Augmentation File that provides data with additional samples for five 
states, researchers can perform state-to-state or state-to-nation comparisons. While the National 
Assessment of Education Progress (NAEP) survey provides cross-sectional assessment data, the 
NELS:88 longitudinal design (following the same cohort of students) enables researchers 
performing state analysis to look at more specific policy factors that may explain the outcome 
differences within or across states. The results of such studies may go beyond the scope of an 
aggregated "report card" and pinpoint problems contributing to equity in specific states. Data 
available from state governments may be merged with NELS:88 data to expand the range of 
research issues. 

Caution: Specific sampling weights may be needed for a given state; the augmentation file is 
restricted-use data and only available through a licensing procedure (contact the NCES Data 
Security Officer). 
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Relevant literature 



Connecticut State Department of Education. (1989). Connecticut participation in the 
National Education Longitudinal Study (NELS). Connecticut State Department of Education 
Research Bulletin , no. 3. 

New York State Education Department. (1991). Characteristics of New York State’s 
eighth grade students from the National Education Longitudinal Study of 1988 . Albany, NY: 
Author. 



Issue: Schooling of students with disabilities or other special needs 

Federal laws mandate a provision of equal access to education for children with special needs 
caused by mental/physical disabilities. Limited English proficiency (LEP) students also need 
special help to ensure their equal access to education. Much investment has been made to serve 
these special groups at different levels of administration. However, the actual quality of education 
received by these children and the effectiveness of the special programs has not been fully 
examined at the national level. 

NELS:88 provides data to address the issues from different perspectives: students, parents, and 
school administration. A systematic examination of access to and quality of education for disabled 
children is feasible with NELS: 88 data. 

Caution: Exclusion of severely disabled students (especially those with learning disorders or 
retardation) or LEP students from the NELS:88 sample limits the coverage of special needs 
students. 

Relevant literature 



Bradby, D. (1992). Language characteristics and academic achievement: A look at Asian 
and Hispanic eighth graders in NELS:88. Statistical analysis report . NCES 92-479. Washington, 
DC: NCES. 

Hodapp, R.M. & Drasner, D.V. (1994-95). Reflections on using large, national data bases 
in special education research. Exceptionality. 5 (2). 103-08. 

Waggoner, D. (Ed.). (1991). Numbers and needs: Ethnic and linguistic minorities in the 
United States. The Number News. 1-2 . Education Resource and Information Center Document 
No. ED 359285. 



3. At-risk Students 

Issue: The characteristics of students who are at risk of school failure 

The most obvious at-risk students are high school dropouts. Hispanic dropouts are especially 
poorly understood in research. For example, what is the effect of Hispanic immigrant families’ 
mobile lifestyle on school performance and dropout rates? Other at-risk groups include students 
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who are involved in violence, substance abuse, teen sex/parenthood, and gang activity. Multiple 
disadvantageous factors often affect the same demographic group (e.g., for urban black children, 
poverty, single-parent family, crime-ridden community, and youth violence frequently happen 
together). 

Since NELS:88 oversampled Hispanic students, it can be used to closely examine this population. 
With student reported risk behavior and data on disadvantaged background, the relationships 
between student characteristics and risk behaviors can be specified. 

Caution: It should be pointed out that 5 percent of the base year students were excluded from 
the study because of physical, mental, and language problems. Many of these students could be 
considered at-risk. 

Relevant literature 



Burbridge, L.C. (1991). The Interaction of race, gender, and socioeconomic status in 
education outcomes. Center for Research on Women Working Paper Series. No. 246 . Paper 
presented at the annual meeting of the American Sociological Association, Cincinnati, August 
1991. Education Resource and Information Center Document No. ED 360243. 

Collins, M.A. (1992). School social context, self-esteem, and locus-of-control among 
white, black, and Hispanic youth . Unpublished doctoral dissertation. University of Maryland at 
College Park. 

De La Rosa, D. & Maw, C.E. (1990). Hispanic education: A statistic portrait 1990 . 
Washington, DC: National Council of La Raza. 

Finn, J.D. (1993). School engagement and students at risk . NCES 93-470. Washington, 
DC: NCES. 

Hafner, A., Ingels, S.J., Schneider, B. & Stevenson, D.L. (1990). A Profile of the 
American eighth grader: NELS:88 student descriptive summary . NCES 90-458. Washington, DC: 
NCES. 

Hawkins, R. et al. (1992). Athletic investment and academic resilience among African- 
American females and males in the middle grades . Cleveland, OH: Cleveland State University. 
Education Resource and Information Center Document No. ED 361450. 

Hawkins, R., Royster, D.A. & Braddock, J.H. (1992). School sports and academic 
resilience among African American. Anglo, and Latino males . Paper presented at the Meeting of 
the American Sociological Association, Pittsburgh, August 1992. 

Hernandez-Gantes, V.M. (1993). Direct and indirect influences of school learning variables 
on Hispanic American eighth-grade students’ academic achievement . Unpublished doctoral 
dissertation. Virginia Polytechnic Institute and State University. 

Hispanic Policy Development Project, Inc. (1991). Disadvantaged urban eighth graders: 
Where they are and how they do. Research Bulletin. 2 0). Education Resource and Information 
Center Document No. ED 335431. 

Ingels, S.J. & Scott, L.A. (1991). National dropout statistics from a longitudinal cohort 
perspective: Estimating rates of school-leaving and school noncompletion . Paper presented at the 
meeting of the American Educational Research Association, Chicago, IL, April 1991. Education 
Resource and Information Center Document No. ED 334260. 
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Jordan, W.J., Lara, J. & McPartland, J.M. (1994). Exploring the complexity of early 
dropout causal structures . Research report no. 48. Baltimore, MD: Center for Research on 
Effective Teaching for Disadvantaged Students. 

Kaufman, P. & Bradby, D. (1992). Characteristics of at-risk students in NELS:88 . NCES 
92-042. Washington, DC: NCES. 

Krasner, D. (1992). Risk and protective factors and achievement of children at risk . 
Unpublished doctoral dissertation. University of California at Los Angeles. 

Peng, S.S. & Lee, R.M. (1991). Educational experiences and needs of middle school 
students in poverty. In K.K. Wong & M.C. Wang (Eds.), Rethinking policy for at-risk students . 
Berkeley, CA: McCutchon Publishing Co. 

. (1992) Home variables, parent-child activities, and academic achievement: A 

study of 1988 eighth graders . Paper presented at the meeting of the American Educational 
Research Association, San Francisco, CA, April 1992. 

. (1992). Measuring student at-riskness by demographic characteristics . Paper 

presented at the annual meeting of the American Educational Research Association, San 
Francisco, CA, April 1992. 

Peng, S.S., Wang, M.C. & Walberg, H.J. (1992). Demographic disparities of inner-city 
eighth graders. Urban education. 26 (4). 441-459. 

Peng, S.S., Wright, D. & Hill, S.T. (1995). Understanding racial-ethnic differences in 
secondary school science and mathematics achievement. Research and development report . NCES 
95-710. Washington, DC: NCES. 

Ukaga, O.M. (1993). Structural determinants of at-risk students: relationship with eighth 
graders’ expectations for completing high school . Unpublished doctoral dissertation. Pennsylvania 
State University. 



Issue: The effectiveness of school programs in retaining teen mothers or attracting them 
back to school 

Teenage motherhood is an established risk factor closely associated with high school dropouts. 
The combination of the two experiences often leads to a life-course failure. A relevant policy 
issue is to identify school programs that are effective in retaining teen mothers or attracting them 
back to school so that they can finish high school and eventually function as productive 
individuals. Program factors (alternative programs such as within a high school or with GED 
programs) together with family and individual efforts, contribute to teen mothers’ school 
completion. 

The issue may be tackled with NELS:88 data that cover both teen pregnancy and school dropout. 
With data collected in the third followup (conducted in 1994 and currently under processing), 
research can follow dropout teen mothers to figure out their whereabouts in relation to special 
services they received. 

Caution: Information is not complete for all dropouts in followup surveys. For example, the 
NELS:88 Second Followup surveyed a portion of students who were identified as dropout and 
alternative students with student questionnaires rather than dropout questionnaires. Thus, in the 
resultant responses, information on the timing of dropping out or of returning to school is absent. 
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Relevant literature 



Ingels, S.J. & Scott, L.A. (1991). National dropout statistics from a longitudinal cohort 
perspective: Estimating rates of school-leaving and school noncompletion . Paper presented at the 
meeting of the American Educational Research Association, Chicago, IL, April 1991. Education 
Resource and Information Center Document No. ED 334260. 

Jordan, W.J., Lara, J. & McPartland, J.M. (1994). Exploring the complexity of early 
dropout causal structures . Research report no. 48. Baltimore, MD: Center for Research on 
Effective Teaching for Disadvantaged Students. 



Issue: Student misbehaviors and school disciplining measures 

Student misbehavior in relation to academic learning and achievement is an issue relevant to the 
research of at-risk students. Student behavior normally is highly stable (e.g., disciplining, 
engagement) and impacts academic outcomes. Student misbehaviors (including absence or 
truancy, violence, drug use, and crime) can be studied to identify community and school 
conditions which may be responsible for behavioral problems or that contribute to school order. 
It may be also interesting to examine the relationship between misbehavior and other psychosocial 
problems such as impaired self-concept and emotional disorder (see the issues in number 2. Self- 
Concept, Locus-of-Control, and Social Behavior under Motivation, Aspiration, Self-Concept, and 
Other Psychosocial Issues). For both practitioners and policymakers, knowledge on these issues 
will be valuable. 

NELS:88 provides data on problem behavior and school discipline measures (reported by 
students) which can be linked to school performance; the school survey contains data on school 
policies and efforts for discipline which can be related to student reported misbehavior. 

Relevant literature 



Braddock, J.H.II & Slavin, R.E. (1992). Why ability grouping must end: Achieving 
excellence and equity in American education . Paper presented at the Common Destiny 
Conference, Johns Hopkins University, Baltimore, MD, September 1992. Education Resource and 
Information Center Document No. ED 355296. 

Dewalt, M.W. (1992). The relationship between youth group participation and use of 
tobacco and drugs . Paper presented at the meeting of the American Educational Research 
Association, San Francisco, CA, April 1992. Education Resource and Information Center 
Document No. ED 350375. 

. (1993). Using youth group participation variables and smoking to predict drug 

use in a national sample . Paper presented at the meeting of the American Educational Research 
Association, Atlanta, April 1993. Education Resource and Information Center Document No. ED 
363558. 

Spiegel, A.N. (1994). Student and school variables related to the academic engagement 
of at-risk students . Unpublished doctoral dissertation. University of Minnesota. 

Weishew, N.L. & Peng, S.S. (1993). Variables predicting students’ problem behaviors. 
Journal of Educational Research. 87 0!. 5-17. 
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Issue: The impact of retention on student performance 

Controversial as it is, retention is a measure intended to, by repeating the grade, help students 
meet academic achievement requirements after they fail at the current grade level. Critics argue 
that this approach works against its intended goal: instead of helping retained students, it 

damages students’ self-esteem and academic motivation, and makes these students fall further 
behind. Because retention happens more often to disadvantaged students, the joint effect of 
retention and other risk factors may push students to eventual school failure. Many issues in this 
connection, however, remain unclear. For example, is there a uniform negative effect of retention 
on later achievement? If not, what are the differential effects of retention on different student 
groups (e.g., gender, race, or students with specific disabilities)? Does the timing of retention 
(e.g., in middle school versus high school) make differences to the outcomes? What are 
alternative ways to help students who temporarily fail in school? 

Two types of information on retention are available in NELS:88. Transcript data from eighth 
grade to high school senior reflect retention experiences in middle school and high school. Parent 
recollection of experience of retention at each grade level from kindergarten to seventh grade is 
another source for checking early retention. With data on student background and school 
outcomes, a variety of retention related issues can be addressed. 

Caution: Retrospective data collected from students or parents on early experiences of retention 
should be handled with caution, and perhaps be checked against transcript data. 

Relevant literature 



Meisels, S. J. & Liaw, F.R. (1993). Failure in grade: Do retained students catch up? 
Journal of Educational Research, 87 (2), 69-77. 



Issue : Academic resilience and school experience among disadvantaged students 

Not all disadvantaged children fail in school. Some of them are highly successful in secondary 
and college education and many manage to acquire basic literacy and to function in society. For 
practitioners and researchers alike, it is interesting to understand how academic resilience is 
developed against the odds of a disadvantageous environment. Children of racial or language 
minorities, families in poverty, and schools in inner cities or remote rural areas, for example, may 
be considered at education disadvantage. Studies are needed to learn about the process in which 
institutional, community, and family conditions work with personal characteristics to develop 
resilience and academic success. These conditions may interact with one another and jointly 
contribute to subsequent school performance and personal development. 

With the NELS:88 oversampling of certain disadvantaged groups (Hispanic and Asian students), 
a reasonably large number of resilient students can be identified for analysis. Drawing data about 
school, family, and student personal experience, NELS:88 makes it feasible to examine the 
resilience issue. Attention should be focused on the interaction patterns in which school, family, 
and the student’s personal effort affect outcomes. 
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Relevant literature 



Finn, J.D. (1993). School engagement and students at risk . NCES 93-470. Washington, 
DC: NCES. 

Hawkins, R., Royster, D.A. & Braddock, J.H. (1992). School sports and academic 
resilience among African American. Anglo and Latino males . Paper presented at the Meeting of 
the American Sociological Association, Pittsburgh, August 1992. 

Krasner, D. (1992). Risk and protective factors and achievement of children at risk . 
Unpublished doctoral dissertation. University of California at Los Angeles. 

Peng, S.S. & Hill, S.T. (1994). Characteristics and educational experiences of high- 
achieving minority secondary students in science and mathematics. Journal of Women and 
Minorities in Science and Engineering. L 89-104. 

Peng, S.S. & Lee, R.M. (1991) Diversity of Asian American students and its implications 
for education: A study of 1988 eighth graders . Philadelphia, PA: Temple University. 

Peng, S.S., Wang, M.C. & Walberg, H.J. (1992). Resilient students in urban settings . 
Paper presented at the annual meeting of the American Educational Research Association, San 
Francisco, CA, April 1992. 

Peng, S.S. & Wright, D. (1994). Explanation of academic achievement of Asian American 
students. Journal of Educational Research, 87 (6), 346-352. 

Peng, S.S., Wright, D. & Hill, S.T. (1995). Understanding racial-ethnic differences in 
secondary school science and mathematics achievement. Research and development report . NCES 
95-710. Washington, DC: NCES. 

Wood, D.D. (1993). Academic achievement of eighth-grade minority students: path 
analysis of Asian. Hispanic, and black eighth-graders . Unpublished doctoral dissertation. 
University of South Florida. 

Zhang, M. (1993). Family as a dynamic of educational attainment: Asian students in 
America . Unpublished doctoral dissertation. University of South Carolina. 



Issue: The causes and the long-term effects of teen parenthood 

Teen sex, pregnancy, and parenthood not only affect students’ academic achievement but also 
affect their adult lives. In connection to the proposed welfare reform, research into the causality 
and the long-term effects of teen parenthood is highly significant. For example, what is the 
impact of teen pregnancy on high school completion and postsecondary educational attainment? 
How effective are programs of sex education and pregnancy prevention in reducing the problem? 
What is the effect of family/parental involvement in both preventing and coping with the 
eventuality? What kind of role do peers play in the process? 

NELS:88 systematically collects data on dating, sexual behavior (e.g., first time sex, the number 
of partners, and use of contraception), knowledge about sex and pregnancy, pregnancy, and child 
bearing. Data on school programs dealing with teen sex and pregnancy are available as well. The 
longitudinal design makes it possible to examine the effectiveness of programs and the 
consequence of teen pregnancy. 
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4. Process of Education Reform 

Issue: The impact of school restructuring on teaching and learning 

School restructuring has been a continuing approach to improving education. Research has not, 
however, systematically assessed the consequences of school restructuring in connection to 
achievement outcomes. School restructuring calls for flexible school organization, decisionmaking 
participation by teachers and parents, heterogeneous grouping, and team teaching. With such 
changes, student engagement and academic achievement are expected to improve. While school 
restructuring is underway to foster a community of learning, it also poses new dilemmas to 
teachers. 

NELS:88 contains some relevant data collected from teachers, administrators, and students, 
including departmentalization, classroom grouping, and teaching responsibility and organization, 
among others. Researchers need to carefully conceptualize school restructuring and build 
appropriate measures to tackle the complicated processes of changes and link the changes to 
school performance. 

Relevant literature 



Lee, V.E. & Smith, J.B. (1992). Effects of school restructuring on the achievement and 
engagement of middle-grade students. Sociology of Education. 66 (31. 164-187. 

Lee, V.E. & Smith, J.B. (1995). Effects of high school restructuring and size on early 
gains in achievement and engagement. Sociology of education. 68 (4), 241-270. 

Lee, V.E. & Smith, J.B. (1994). High school restructuring and student achievement. A 
new study finds strong links. Issues in restructuring schools. 7 (Fall). Madison, WI: Center on 
Organization and Restructuring of Schools. 

Lee, V.E. & Smith, J.B. (1994). Effects of high school restructuring and size on gains in 
achievement and engagement for early secondary school students . Madison, WI: Center on 
Organization and Restructuring of Schools. 

Office of Educational Research and Improvement. (1990). Restructuring schools for young 
adolescents. Issues in education . OERI-AS-90-002. Washington, DC: Author. 

Taylor, D.L. & Tashakkori, A. (1994). Predicting teachers’ sense of efficacy and job 
satisfaction using school climate and participatory decision making . Paper presented at the 
meeting of the Southwest Educational Research Association, San Antonio, TX, January 1994. 



Issue: Academic achievement gains in the high school years: intercohort comparison 
between the HS&B and NELS:88 cohorts 

Current education improvement can benefit from historic comparisons of academic performance 
resulting from education improvement efforts. For example, we may examine the academic 
achievement gaps across curriculum groups and student backgrounds. Significant fluctuations of 
such gaps related to policy changes will be informative to the present education reform. 
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NELS:88 and HS&B contain many comparable measures of academic achievement, student 
demographics, and grouping by program/curriculum. Such comparability enables intercohort 
analysis of achievement gap relating to SES background and curriculum grouping. The analysis 
can clarify whether or not the achievement gaps between curriculum groups and demographic 
groups have widened in the last decade when education reform has been on the national agenda. 

Group differences in learning on specific subjects (e.g., math and science) for the same grade 
span (e.g., from sophomore to the senior year) of the two cohorts, measured with standard 
deviation units, can be compared. A partition analysis used by demographers can partition cohort 
differences into three components: difference due to demographic shifts, difference due to 

educational reform (e.g., policies and curriculum), and difference due to the interaction of the 
two. 



Issue: Studying systemic reform of school with econometric modeling 

Education reform calls for schools to produce high quality graduates with limited resources. To 
achieve school effectiveness, administrators and policymakers need to understand the process by 
which variables of involved parties (the district, school, teachers, and students) jointly determine 
the final outcomes (i.e., students of high achievement and responsible citizenry). An econometric 
modeling approach can be applied to scrutinize the factors operating in the reform. 

NELS:88 with its comprehensive coverage of relevant data and longitudinal design makes 
econometric modeling possible. Such models address learning environment/ process (including 
both classroom conditions and broader institutional reform) and performance outcomes. This 
approach not only involves multilevel modeling (such as in HLM) but also allows simultaneous 
estimating to predict the outcomes with multilevel path analysis. Some ongoing studies are 
already attempting this: with teacher- student paired data and school-level data, by modeling 
science teaching and learning, and with simultaneous three-level path analysis to predict science 
education outcome. With "switches" in the model (i.e., variables that are important theoretically 
or closely related to policymaking and that can be manipulated in the analysis), one can use the 
model to see how these variables affect the outcome and use the models for decision-making. 

Caution: A constraint to such analyses with NELS:88 data is that the sample did not have 
enough students in each class, nor enough classes in each school. It is only possible to model two 
levels, the student and the school. Aggregating classroom data into the school level may lessen 
or confuse the effects of instruction/curriculum variables. 

Relevant literature 



Akerhielm, K.B. (1994). The determinants of student achievement in the United States: 
A microeconometric study with endogenous educational inputs . Unpublished doctoral dissertation. 
Yale University. 



28 ~ N ELS: 88 Research Framework and Issues 



3 ? 



Secondary Education 



Issue: The effectiveness of restructured social services in the school and community ' 

Integrated services is a new strategy for dealing with multiple problems facing disadvantaged 
students and families. How has this approach worked so far? With what kind of problems or 
student groups has integrated services worked more effectively? Answers to these questions are 
helpful for improving practice and decisionmaking on program administration and planning. 

With data from recent years, NELS:88 may be able to help address some of the questions. 



Issue: The impact of national or state standards for academic achievement 

The notion of national standards that hold high expectations for all children are controversial. 
Critics say that standardization simply adds bureaucratic procedures to the already ineffective 
system and forces students with diverse backgrounds to learn skills irrelevant to the needs of the 
local labor market and community. Empirical evidence is required to show whether high 
standards actually help schools raise their performance through accountability. 

NELS:88 data allow such research. Outcome measures are available for students with different 
prior performance. Student and school background, resources, technical support, and incentives 
for achieving the standards also can be examined to isolate the effects of standardization. Study 
can compare the achievement outcomes of schools that have adapted state standards or some form 
of national standards and those that have not. 

Caution: Data on state standards and on school adaptation of standards may be insufficient. 
Additional data are probably needed from state governments. 

Relevant literature 



Office of Education Research and Improvement. (1994). What do student grades mean? 
Differences across schools . Washington, DC: Author. 

Thomas, J. (1990). The numbers game: What eighth graders study — and how much of it 
they do. Humanities, 11 (6). 9-10. 



5. Parent/Communitv Involvement 

Issue: The functions of market mechanisms and parents ’ "choice" of school that increase 
school competition for student enrollment 

Research is needed to find out how to empower parents so that they can exercise real influence 
in school policymaking and operation. Some market mechanisms have been introduced to increase 
the competition for students among schools in order to improve school efficiency. Parents’ 
"choice" of school is a key element in the proposed change. Has this competition mechanism 
worked to strengthen parents’ influence and to improve school accountability? Many are skeptical 
about this. Critics argue that such changes are hurting children of disadvantaged families and may 
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increase inequality in the system. Scientific studies are needed to inform the ongoing debate on 
this issue. NELS.88 provides most of the relevant data needed for such studies. 

Caution: Data about specific market mechanisms may be inadequate. Other data sources may 
need to be incorporated. 

Relevant literature 



Coleman, J.S., Schiller, K.S. & Schneider, B. (1993). Parent choice and inequality. In B. 
Schneider & J.S. Coleman tEds.T Parents, their children, and schools (pp. 147-1 82). Boulder, CO: 
Westview Press. 

Goldhaber, D.D. (1994). Public and private high schools: School choice and the 
consequences . Unpublished doctoral dissertation. Cornell University. 

Sosniak, L.A. & Ethington, C.A. (1992). When public school “choice” is not academic: 
Findings from the National Education Longitudinal Study of 1988. Educational Evaluation and 
Policy Analysis. 14 (1), 35-52. 



Issue: Forms of parental involvement at home and with the school, the conditions under 
which different forms of involvement take place, and the effects of different forms 
of involvement on student performance 

It is abundantly clear that parents play a crucial role in children’s education. It is not clear, 
however, what kind of parental involvement works better for different student groups. For 
instance, expectations and emotional support in contrast to actual help to student learning in forms 
of discussion, mentoring, and examining results both seem necessary. But not all parents are 
making such efforts. Different parent roles in children’s schooling, while often rooted in cultural 
traditions, are subject to changes under the influence of school efforts to engage parents. The 
extent to which different parental approaches succeed also depends on student characteristics. 
Knowledge about the effectiveness of parental support and how to advise parents to use 
appropriate approaches can help schools develop better strategies for involving parents. 

With NELS:88 data gathered from students, teachers, and parents, in-depth study into parental 
involvement is possible. 

Relevant literature 



Coleman, J.S., Schiller, K.S. & Schneider, B. (1993). Parent choice and inequality. In 
Schneider, B.& Coleman, J.S. (Eds.). Parents, their children, and schools (pp. 147- 182V Boulder, 
CO: Westview Press. 

Keith, P.B. (1992). Effects of parental involvement on Mexican-American eighth-grade 
students’ academic achievement: A structural equations analysis . Unpublished doctoral 
dissertation. Virginia Polytechnic Institute and State University. 

Keith, T.Z., Keith, P.B., Troutman, G.C., Bickley, P.G. et al. (1993). Does parental 
involvement affect eighth-grade student achievement? Structural analysis of national data. School 
Psychology Review. 22 (31. 474-496. 
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Kerbow, D. & Bernhardt, A. (1993). Parental intervention in the school: The context of 
minority involvement. In B. Schneider & J.S. Coleman (Eds.), Parents, their children, and schools 
(pp.l 15-146). Boulder, CO: Westview Press. 

Muller, C. (1993). Parental involvement in education and school sector . Paper presented 
at the annual meeting of the American Educational Research Association, Atlanta, GA, April 
1993. 

. (1993). Parental involvement and academic achievement: An analysis of family 

resources available to the child. In B. Schneider & J.S. Coleman (Eds.), Parents, their children, 
and schools (pp. 77-1 13). Boulder, CO: Westview Press. 

. (1991). Parental involvement in the education process: An analysis of family 

resources and academic achievement. Unpublished doctoral dissertation. University of Chicago. 

Muller, C. & Kerbow, D. (1993). Parent involvement in the home, school, and 
community. In B. Schneider & J.S. Coleman (Eds.), Parents, their children, and schools (pp.13- 
42). Boulder, CO: Westview Press. 

Muller, C. et al. (1991). Defying statistics or "latch-key" children in the late ’80s: Family 
composition, working mothers, and after school supervision . Paper presented at the annual 
meeting of the American Educational Research Association, Chicago, IL, April 1991. 

Office of Educational Research and Improvement. (1990). Parental involvement in 
education. Issues in education . Washington, DC: Author. 

. (1992). Parental satisfaction with schools and the need for standards. Education 

research report . Washington, DC: Author. 

Peng, S.S. & Wright, D. (1994). Explanation of academic achievement of Asian American 
students. Journal of Educational Research. 87 (6). 346-352. 

. (1993). Learning programs at home: An explanation of the high academic 

achievement of Asian American students . Paper presented at the meeting of the American 
Statistical Association, Ft. Lauderdale, FL, January 1993. 

West, J., Rasinski, K.A. & Camburn, E. (1990). Parental involvement in education: 
Findings from the NELS.88 base-year parent survey . Paper presented at the meeting of the 
American Education Research Association, Boston, MA, March 1990. 



Issue: Home environment and family support for learning in relation to student 
motivation, learning approach, and education outcomes 

The dilemma facing public schools in teaching values and behavior leaves family a major role 
to play in guiding and supporting students to learn. Available research suggests that the family 
helps form children’s values, aspirations, and expectations that are essential for achieving 
academic excellence. These influences, as cultural ingredients, are exerted in the home setting by 
senior members of the family. Understanding how home environment and school factors jointly 
affect student learning and behavior can contribute to policymaking and program development 
for parental involvement. 

NELS:88 contains a wealth of data for scrutinizing home environment; for example, family 
composition and family members (parents, siblings, and other relatives), home language, family 
rules regarding children’s behavior and activities, parents’ expectations for children, interaction 
between parents and children, family activities conducive to learning and achievement (e.g.. 
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lessons or programs outside of school, visiting libraries and other cultural facilities), and family 
resources for learning (e.g., place for study, computer, reading materials). Indicators of home 
environment can be constructed to identify subgroup differences in home environment. 

Caution: Data provided by the student may differ from that by the parents. Validation and 

integration of data from different files is advisable. 

Relevant literature 



Cagampang, F.H.H. (1992). What parents know about preparing for college and how it 
affects their children’s academic performance: Parents’ information in the college choice decision . 
Unpublished doctoral dissertation. University of California at Berkeley. 

Coleman, J.S., Schiller, K.S. & Schneider, B. (1993). Parent choice and inequality. In B. 
Schneider & J.S. Coleman (Eds.), Parents, their children, and schools (pp. 147-182). Boulder, CO: 
Westview Press. 

Conway, K.P. (1994). Parent involvement, gender and race/ethnicitv: The relationship of 
student and self-concept . Unpublished doctoral dissertation. Hofstra University. 

Downey, D.B. (1995). Understanding academic achievement among children in 
stephouseholds: The role of parental resources, sex of stepparent, and sex of child. Social forces, 
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of education reform. Paper presented at the meeting of the Eastern Psychological Association, 
Boston, MA, April 1992. 
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predictors of reading comprehension in eighth-grade students . Unpublished doctoral dissertation. 
University of Tennessee at Nashville. 

Finn, J.D. & Owings, M.F. (1994). Family structure and school performance in eighth 
grade. Journal of Research and Development in Education. 27 (31, 176-87. 

Keith, P.B. (1992). Effects of parental involvement on Mexican-American eighth-grade 
students’ academic achievement: A structural equations analysis . Unpublished doctoral 
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Keith, P.B. & Lichtman, M.V. (1994). Does parental involvement influence the academic 
achievement of Mexican-American eighth graders? Results from the National Education 
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Lee, V.E., Burkam, D.T., Zimiles, H. & Ladewski, B. (1994). Family structure and its 
effect on behavioral and emotional problems in young adolescents. Journal of Research on 
Adolescence. 4 (3'). 405-437. 

Muller, C. (1993). Parental involvement in education and school sector . Paper presented 
at the annual meeting of the American Educational Research Association, Atlanta, GA, April 
1993. 

Peng, S.S. & Lee, R.M. (1992) Home variables, parent-child activities, and academic 
achievement: A study of 1988 eighth graders . Paper presented at the meeting of the American 
Educational Research Association, San Francisco, April 1992. 

Zhang, M. (1993). Family as a dynamic of educational attainment: Asian students in 
America . Unpublished doctoral dissertation. University of South Carolina. 



Issue: The influence of community participation in school decisionmaking on school 

performance 

The role of a caring community is believed to be a key factor affecting education improvement. 
How does community participation in school decisionmaking help improve school performance? 
Do children have a sense of attachment to the community? The community or students’ 
immediate neighborhood may influence students’ behavior, aspirations, and educational 
performance to a great extent. And such community influences are particularly intense in 
disadvantaged urban neighborhoods and isolated rural places. However, the community-school 
relationship and its impact on school outcomes is probably one of the weakest areas in education 
research. 

Merging NELS:88 data with Census and CCD data offers opportunities for dealing with these 
issues. The research can be oriented to developing both policy and theory. 

Caution: The relatively thin data on neighborhood in NELS:88 may require additional efforts 
to incorporate appropriate Census data or locally collected information. 



6. Student Mobility 

Issue: Student transition: continuing versus dropping out of school, transferring across 
schools or school sectors, and other issues related to student mobility 

Over the years of secondary schooling, especially between middle school and high school, a 
substantial portion of students experience transitions that often cause stress and influence students’ 
later education: early dropping out can happen during the transition, some students change from 
public to private schools or vice versa, and most students change their schools as they upgrade 
from middle school or junior high into high school. Some families move and students are 
relocated into different schools. Administration imposed transition may take place in such forms 
as school consolidation and district rezoning. Learning about the complicated patterns of student 
transition and subsequent school experience is helpful for improving administration and planning 
to reduce difficulties facing mobile students. 



NELS. 88 Research Framework and Issues 33 



Summary of Research Issues 



NELS:88 tracks students’ transitions; the resulting high sample attrition rates provide data for 
such studies. 

Relevant literature 



Owings, J. & Peng, S. (1992). Transitions experienced by 1988 eighth graders. Statistics 
in brief . NCES 92-023. Washington, DC: NCES. ~ 

Scott, L.A. & Ingels, S.J. (1992). Transitions to high school; instruction and achievement: 
Findings from the NELS:88 first followup (1990) student survey . Paper presented at the annual 
meeting of the American Educational Research Association, San Francisco, CA, April 1992. 

Scott, L.A., Rock, D.A., Pollack, J.M. & Ingels, S.J. (1995). Two years later: Cognitive 
gains and school transitions of NELS:88 eighth graders . Washington, DC: NCES. 



7. School Organization and Climate 

Issue: School organization, administration, and leadership in connection to education 

outcomes 

The school is a social organization with diverse administrative structures, varying sizes of staff 
and enrollment, changing policies, and distinct institutional cultures. It is a challenge to education 
research to understand how school works to achieve equity and excellence. Systematic studies 
focusing on school as an institution are framed in different intellectual perspectives (e.g., in 
economic models, school is an input/output production process; in sociological theories, school 
is a structured arrangement wherein groups and individuals interact). Relevant to policy analysis, 
school sector (public versus private), size, socioeconomic conditions of the community, 
administrative approach, policies and climate are all important concepts. 

NELS:88 offers school data including school demographics (e.g., enrollment, student/staff 
racial/ethnic composition, sector, and locale), school facilities, school regulations and rules, 
program offerings, and school climate as perceived by teachers and students. Data were collected 
from administrators, teachers, parents and students. Linking data about school resources and 
climate with data on student performance, economic models can be applied. Linking different 
sources of data about school culture (e.g., students’ perception about school climate and student- 
teacher relationships, teachers’ perception about student behavior and school administration), 
social psychological concerns can be addressed. 

While NELS:88 has not maintained its school sample as nationally representative after the base 
year, a School Effectiveness Study was added to the first followup to provide a nationally 
representative sample of 10th and twelfth grade schools. This supplementary project sustains 
longitudinal school-level analysis. 

Caution: In followups, schools are no longer a focus of sampling design because of complicated 
student transitions after middle school. Therefore, school weighting needs to be developed (see 
the first issue of number 2, Research on Sampling Problems, in Methodological Research) if 
school is the unit of analysis. School weights were developed for the School Effectiveness 
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Supplement data. This data can only be obtained, though, through a licensing procedure. (Contact 
the NCES Data Security Officer.) 

Relevant literature 



Akerhielm, K.B. (1994). The determinants of student achievement in the United States: 
A microeconometric study with endogenous educational inputs . Unpublished doctoral dissertation. 
Yale University. 

Akst, D. (1993). Looking at what works — the City’s Catholic school system: Interview 
with Jerome Porath. Los Angeles Times , (14 March 1993): Home edition, p.3. 

Anderson, J. et al. (1992). Poverty and achievement: Re-examining the relationship 
between school poverty and student achievement . Paper presented at the meeting of the American 
Education Research Association, San Francisco, April 1992. Education Resource and Information 
Center Document No. ED 346207. 

Collins, M.A. (1992). School social context, self-esteem, and locus-of-control among 
white, black, and Hispanic youth . Unpublished doctoral dissertation. University of Maryland at 
College Park. 

Conway, G.E. (1994). Small scale and school culture: The experience of private schools. 
ERIC Digest . Charleston, WV: ERIC Clearinghouse on Rural Education and Small Schools. 

Crone, L.J. & Tashakkori, A. (1992). Variance of student achievement in effective and 
ineffective schools: Inconsistencies across SES categories . Paper presented at the meetings of the 
American Educational Research Association, San Francisco, CA, April 1992. Education Resource 
and Information Center Document No. ED 346613. 

Eccles, J.S. et al. (1991). What are we doing to early adolescents: The impact of 
educational contexts on early adolescents. American Journal of Education. 99 (41. 521-542. 

Finn, J.D. & Voelkl, K.E. (1993). School characteristics related to student engagement. 
Journal of Negro Education. 62 (31. 249-68. 

Goldhaber, D.D. (1994). Public and private high schools: School choice and the 
consequences . Unpublished doctoral dissertation. Cornell University. 

Hoachlander, E.G. (1991). A profile of schools attended by eighth graders in 1988. 
Statistical analysis report. NELS:88 . NCES 91-129. Washington, DC: NCES. 

Lee, V.E. & Smith, J.B. (1992). Effects of school restructuring on the achievement and 
engagement of middle-grade students. Sociology of Education. 66 (3). 164-187. 

Lee, V.E. & Smith, J.B. (1994). Effects of high school restructuring and size on gains in 
achievement and engagement for early secondary school students . Madison, WI: Center on 
Organization and Restructuring of Schools. 

Lee, V.E. & Smith, J.B. (1995). Effects of high school restructuring and size on early 
gains in achievement and engagement. Sociology of education. 68 (41. 241-270. 

Lee, V.E. & Smith, J.B. (1994). High school restructuring and student achievement. A 
new study finds strong links. Issues in restructuring schools. 7 (Fall). Madison, WI: Center on 
Organization and Restructuring of Schools. 

Schneider, B. & Shouse, R. (1992). Children of color in independent schools: An analysis 
of the eighth-grade cohort from the National Education Longitudinal Study of 1988. Journal of 
Negro Education. 61 (21. 223-234. 

Spiegel, A.N. (1994). Student and school variables related to the academic engagement 
of at-risk students . Unpublished doctoral dissertation. University of Minnesota. 
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Taylor, D.L. & Tashakkori, A. (1994). Predicting teachers’ sense of efficacy and job 
satisfaction using school climate and participatory decision making . Paper presented at the 
meeting of the Southwest Educational Research Association, San Antonio, TX, January 1994. 

Wu, Shi Chang. (1992). Education and learning in schools with high dropout rates. 
Evaluative report . Washington, DC: NCES. 

B. Transition from High School to Work 

Transition from high school to work occurs for most students, including a substantial proportion 
of college goers who do not immediately enter college after high school. The transition may 
involve a series of events, such as high school completion, dropping out, or acquisition of 
alternative credentials such as the GED; labor force participation, employment, and occupational 
advance; and participation in adult education or other on-the-job training. Understanding the 
transition process entails studies on such themes as education/career aspirations, information 
access and planning, labor market adjustment, and career decision making. 

Transition data were collected in the NELS:88 third followup in 1994 and are currently being 
processed. While comparable to NLS-72 and HS&B, NELS:88 will allow broader coverage of 
issues relevant to postsecondary issues because other data sources can be integrated into 
NELS:88. For example, occupational attainment studies require data on local economic conditions 
and the labor market context, which are available by merging NELS:88 files with CCD and 
Census data with the common ID numbers. The school level or household level data will also be 
available on CD-ROM. Thus, opportunity for research on postsecondary issues is enhanced 
substantially with NELS:88 data. 



1. High School Completion. GED. and Dropout 

Issue: High school completion/dropout and related individual, school, and 
community factors 

High school completion is a basic requirement for an individual to function in today’s labor 
market. Without a high school credential, one would face significantly higher risk of low-pay 
employment, unemployment, and dependency on public assistance. High school dropping out, in 
some research, is related to not only individuals’ failure in the labor market, but also to 
institutional consequences such as a loss of local or state tax revenue and an increase of public 
service costs. Comparative study of high school completion/dropout rates across demographic 
groups and regions are thus an important undertaking for policymaking purposes. 

NELS:88 has improved the data quality on dropping out of school. All known dropouts were 
sampled with certainty in followups. Data were collected by administering questionnaires in the 
presence of project staff or via telephone interview. With data on school and community, it is 
possible to do comparative analysis of school completion/dropout rates and to link the differences 
to an array of individual, school, and community variables. 
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Caution: To clarify the understanding of the complicated paths to eventual school completion 
requires careful conceptualization and data scrutiny; for example, within-round dropout-returnees 
(also called "stopout," referring to those who dropped out and then returned to school between 
two waves of the survey) might be undetected in the survey data (see section 2 for the 
discussion). Attrition of dropouts among NELS:88 respondents makes it uncertain whether or not 
the remaining dropouts in the sample represent the national population. Lack of information on 
exact timing of events is another barrier to longitudinal analysis. 

Relevant literature 



Kaufman, P. & Bradby, D. (1992). Characteristics of at-risk students in NELS:88 . NCES 
92-042. Washington, DC: NCES. 

Kaufman, P. et al. (1991). Dropout rates in the United States: 1990 . NCES 91-053. 
Washington, DC: NCES. 

. (1992). Dropout rates in the United States: 1991 . Washington, DC: NCES. 

McGrew, K.S. et al. (1994). Matching information in national data collection programs 
to a model of school completion outcomes and indicators. Technical Report 7 . Minnesota 
University, Minneapolis Coll, of Education: National Association of State Directors of Special 
Education. Alexandria, VA. National Center on Educational Outcomes. Minneapolis, MN: Saint 
Cloud State University, MN. 

McMillen, M. (1992). Eighth to tenth grade dropouts. Statistics in brief . NCES 92-006. 
Washington, DC: NCES. 

McMillen, M.M., Hausken, E.G., Kaufman, P., Ingels, S.J., Dowd, K.L., Frankel, M.R. 
& Qian, J. (1993). Dropping out of school: 1982 and 1992. Issue brief . NCES 93-901. 
Washington, DC: NCES. 

McMillen, M.M., Kaufman, P. & Whitener, S.D. (1994). Dropout rates in the United 
States: 1993 . NCES 94-669. Washington, DC: NCES. 

McMillen, M.M. et al. (1993). Dropout rates in the United States: 1992 . NCES 93-464. 
Washington, DC: NCES. 



Issue . The dropout career and postsecondary outcomes 

Tracing the dropout career may help researchers understand the school dropout as an evolving 
life history within the contexts of school, family, and community. Such an understanding can 
shed light for both policymaking and educational practice aimed at preventing and developing 
remedies for dropouts. 

Using NELS:88 data, we may examine specific personal or family events/factors (e.g., child 
bearing) related to dropping out. After dropping out, different paths are possible. Some former 
students may return and complete school, or drop out again; some may receive a Graduate 
Equivalency Diploma (GED) later on; some may enroll in adult education or vocational 
education, join the military, or even go to college; others may fail, (e.g., commit a crime and go 
to jail). One can also look at dropouts’ attitudes and behavior in regard to civic and social 
responsibilities (voting, community involvement) and see how they relate to postsecondary 
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outcomes. A systematic followup of dropouts with NELS:88 data probably can provide sortie 
answers. 

Caution: Again, issues in defining dropouts, sample attrition, and the exact timing of dropout 
processes should be considered. 

Relevant literature 



Braddock, J.H.II & Slavin, R.E. (1992). Why ability grouping must end: Achieving 
excellence and equity in American education . Paper presented at the Common Destiny 
Conference, Johns Hopkins University, Baltimore, MD, September 1992. Education Resource and 
Information Center Document No. ED 355296. 

Ingels, S.J. & Scott, L.A. (1991). National dropout statistics from a longitudinal cohort 
perspective: Estimating rates of school-leaving and school noncompletion . Paper presented at the 
meeting of the American Educational Research Association, Chicago, IL, April 1991. Education 
Resource and Information Center Document No. ED 334260. 

Jordan, W.J., Lara, J. & McPartland, J.M. (1994). Exploring the complexity of early 
dropout causal structures . Research report no. 48. Baltimore, MD: Center for Research on 
Effective Teaching for Disadvantaged Students. 

National Center for Education Statistics. (1995). Dropouts and late completers . NCES 95- 
720. Washington, DC: NCES. 

Ukaga, O.M. (1993). Structural determinants of at-risk students: Relationship with eighth 
graders’ expectations for completing high school . Unpublished doctoral dissertation. Pennsylvania 
State University. 



2. Effects of High School Experience on Later Life Outcomes 

Issue: Compatibility between the knowledge content of school and the knowledge content 
of work 

To what extent in the existing system does the knowledge content of school match the knowledge 
content of work? What are the conditions under which the knowledge content of school and the 
knowledge content of work match better? "Conditions" can be taken to include curriculum, course 
taking, program provision, and school types (public versus private). Specifically, the effects of 
program provision and course taking should be examined in connection to occupational 
attainment. 

Using NELS:88 and HS&B transcripts, course clusters (i.e., the pattern of course taking for 
different student groups) can be examined to see how such clustering influences high school 
outcomes and career success. The concept of student networks in joint course taking is another 
development that can be explored. 
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Issue: Relative preparedness of high school seniors for entering the labor market or 
postsecondary education 

The concept and the measures of proficiency levels can be used to study high school seniors’ 
relative preparedness for employment or college. Such a study can tackle the question of 
knowledge content and help us to understand the relevance of schooling to labor market 
performance. For example, by examining the percentage of students who were at a specific 
proficiency level on specific subjects and who entered specific sectors of the labor market, one 
can assess the relationship between high school preparedness and occupational outcomes. 

NELS.88 data collected in the third followup can be used for such studies. 

Relevant literature 



Ingels, S.J. et al. (1994). A profile of the American high school sophomore in 1990 . 
National Education Longitudinal Study of 1988 . NCES 95-086. Chicago, IL: National Opinion 
Research Center. 

Owings, J.A. et al. (1995). Making the cut: Who meets highly selective college entrance 
criteria? Statistics in brief . NCES 95-732. Washington, DC: NCES. 

Rock, D., Owings, J. & Lee, R. (1994). Changes in math proficiency between eighth and 
tenth grades. Statistics in brief . NCES 93-455. Washington, DC: NCES. 



Issue: Long-term effects of racially integrated schooling (desegregated education) on 
minority students ' college attendance, occupational attainment, and earnings 

Study on the short-term effects of desegregated education (e.g., test results and self-esteem) has 
accumulated a sizable literature, but study on the long-term effects has fallen short. Study on 
long-term effects is essential for assessing the fulfillment of the ultimate goal of school 
desegregation (i.e., overall socioeconomic equity for minorities). 

NELS:88 and prior longitudinal surveys all contain information about school racial composition 
and individual students’ peer racial backgrounds, plus the information on postsecondary outcomes 
collected in the third followup. 

Caution: The definition of a desegregated school requires considering specific conditions such 
as school racial composition, court order, and busing policy. The "self-selection" effect — black 
students who choose to go to desegregated schools may differ in background and behavior from 
other black students — needs to be scrutinized. 



Issue: Problem behaviors in high school and later life conditions 

Problem behaviors such as teen parenthood, substance use, and gang participation may have 
prolonged influence on students’ later life. Educational attainment, employment, and earnings are 
probably subject to the impact of problem behavior in high school. Involvement in adolescent 
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deviant behavior may also contribute to forming social attitudes and value systems in older age. 
There are, however, unsettled issues in this regard. For example, some recent research emphasizes 
that certain risk behaviors (e.g., vandalism and drunk driving) among teenagers are of a 
recreational nature and often diminish in the process of social and psychological maturation. On 
the other hand, it is compelling to assume that problem behaviors are likely to continue if 
socioeconomic or psychoemotional adversities exist in the student’s life. Longitudinal research 
is in order to assess early problem behaviors’ impact on later life conditions. 

NELS:88 contains student reported involvement in problem behavior and later life conditions. It 
is possible to examine problem behaviors such as adolescent violence, drug abuse, teen sex, and 
gang participation to see their relationship with postsecondary education and occupational 
attainment. 



3. Employment and Occupational Attainment 

Issue: Entry into labor force: the rates, timing, and conditions 

A large portion of high school graduates (and nongraduates) enter the labor force without further 
formal schooling after high school. The rates of labor force participation among this population, 
however, differ across geographic areas and student groups. The timing of entry also varies: 
some students start part-time work while still in school, others begin after graduation or after 
further delay. A substantial number of youth do not participate in the labor force after school due 
to child bearing or other difficulties. Finally, the nature of labor force participation differs (e.g., 
temporary job versus permanent career, part-time versus full-time; working while going to school 
versus working without plans for college). Initial wages and working conditions differ as well. 
The differential processes of labor force entrance have important implications for examining and 
crafting education and labor policies. 

NELS:88 contains data on student employment before high school (in the base year to the second 
followup). Data on employment after high school are covered in the third followup. All together, 
there is information about labor force entry, timing, weekly hours, wages, job changes and 
changes in labor force participation status. 



Issue: Employment, occupational performance, and participation in public assistance 

programs 

Policymaking can be informed by studies of the employment experiences of high school graduates 
(and nongraduates) after they enter the labor force. These experiences include occupational 
performance and career advance during employment, changes in income, experience of 
unemployment, and receiving public assistance. Analyses focusing on differences across 
demographic groups or geographic areas will be particularly valuable for advising the current 
reform of the employment and welfare services and other social services programs. Studies 
intended to link employment issues to prior school experience (course work, program 
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participation, and achievement) and school or community factors can also help explain young 
people’s success or failure in the labor market. 

Pertinent data on these issues will be available in NELS:88 third and subsequent followups. 



Issue: Student aspirations and enlistment to military service 

The changing role of the U.S. armed forces (from military confrontations to more humanitarian 
activities) and related downsizing pose problems to military recruitment and enlistment. The 
Department of Defense is trying to understand why and how the recruitment market is shrinking 
and to generate programs to work with the declining market. 

HS&B and NELS:88 both contain items on aspirations to military service, background influences 
on career decision making, and participation in and results of military application tests. A 
combination of background data about family, school, and community with NELS:88 and HS&B 
data is promising for study of this issue. 



Issue: The effect of high school curricula on rural-to-urban migration 

The rural-to-urban migration of educated youth is an important issue in rural communities. Some 
critics contend that the establishment of public schools is actually leading educated rural youth 
to move out of the community and hence contributes to the decline of rural America, because the 
established curriculum is oriented only to the urban labor market. It is not clear what kind of high 
school curriculum can help foster community attachment and construct the meanings of rural life 
as well as teach skills to meet the local needs. The issue is significant not only for rural education 
but also for community development. 

By examining NELS:88 data on curriculum and post-high school migration, the issue can be 
addressed within the context of regional and community economical conditions. 

Caution: Information on local economic conditions is not available readily from NELS:88 files. 
One can, however, merge historical and current data from CCD on community economic 
background into NELS:88 data with common identification numbers. 



Issue: Special groups in the postsecondary transition 

Special groups can be conceived of in many ways: Latinos, women, performing arts students, 
high school athletes, LEP students, the gifted and talented, and disabled students, among others. 
How do these groups do in the postsecondary transition? What are their postsecondary educational 
attainments? What kind of jobs do they get after college? 
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Using NELS:88 (third followup and later), BPS, and B&B data, research can cover issues frOm 
the accessibility of higher education, college persistence and attainment, to postgraduate education 
and job market outcomes. 

Caution: For some special groups, data may lack depth to cover these groups’ unique conditions 
and needs; also, the subsample size may be small. 

Relevant Literature 



Owings, J.A. et al. (1995). Making the cut: Who meets highly selective college entrance 
criteria? Statistics in brief . NCES 95-732. Washington, DC: NCES. 



Issue: The long-term effects of bilingual education and bilingualism 

In the past decades, bilingual education has developed into a large but still controversial 
enterprise. The research has focused mostly on the cognitive/academic aspects of bilingual 
education. The long term social aspects of bilingual education’s impact have been largely ignored. 
With resources invested by school and time spent by bilingual children, what benefit can bilingual 
students accrue in postsecondary education and employment? An assessment of such long-term 
effects in a longitudinal approach is in order. 

NELS:88 measures of family immigration history, home language environment, school bilingual 
programs, oral versus literate bilingual abilities (student reported or transcripts or both) make it 
possible to examine the impact of bilingual education on post high school life. A study may be 
considered with the data on the base-year students excluded due to language barrier. A portion 
of these students were surveyed in followups as their English improved. As these students were 
initially disadvantaged due to different home languages, a study that looks at their postsecondary 
status in relation to their bilingual experience may tell a lot about what works and what does not 
work. 

Relevant literature 



Huang, G.G. (1992). Self-reported biliteracv and self-esteem: A study of Mexican 
American eighth graders . Charleston, WV: Appalachia Educational Laboratory. Education 
Resource and Information Center Document No. ED 356937. 

Kennedy, E. & Park, H.S. (1994). Home language as a predictor of academic 
achievement: A comparative study of Mexican- and Asian- American youth. Journal of Research 
and Development of Education. 27 (31, 188-94. 

Waggoner, D. (Ed.). (1991). Numbers and needs: Ethnic and linguistic minorities in the 
United States. The Number News. 1-2 . Education Resource and Information Center Document 
No. ED 359285. 
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C. Postsecondary Education 

Postsecondary education can be considered in a broad sense as including higher education, 
vocational/technical education, community college, and continuing learning. As such, societal 
changes and institutional factors can affect postsecondary education. 

NELS:88 data from the base year and the second followup has yet to generate much research in 
postsecondary education simply because the majority of cohort members had not entered 
postsecondary cycle before the third followup. The NELS:88 data on postsecondary education 
collected in the third followup of 1 994 are being processed. The results, with census data that can 
be merged with NELS:88, promise great opportunities for research in these areas. 



1. Higher Education Access and Choice 

Issue: The college application process: institution selectivity and student qualifications 
in relation to eventual determination of differential access to higher education 

The unequal distribution of access to higher education is largely determined by two factors: 
institutions’ admission procedures and individuals’ efforts to gain access. Admission procedures 
are regulated by a social structure wherein institutions differentially provide education to different 
student groups. Individuals efforts may be further restrained by their socioeconomic backgrounds 
and social networks. Understanding the interplay of the two factors in affecting college attendance 
may be helpful in improving equity in higher education. 

With NELS:88 third followup data*, we can describe group (e.g., race, sex, and poverty status) 
differences in step-by-step approaches to college entrance (including information gathering, 
application for financial aid, preparation for entrance exams, knowledge of local and federal 
assistance for college education). These descriptions should be linked to analysis of institutional 
processes such as loans and grants provisions, school guidance services, and disadvantaged high 
schools ("outliers"). 

Caution: NELS data on institutional procedures may be less than complete. For additional data, 
analysts may want to consider extracting data from the NCES Integrated Postsecondary Education 
Data System (IPEDS). IPEDS is a single, comprehensive data source that includes all identified 
institutions whose primary purpose is to provide postsecondary education. 

Relevant literature 



Cagampang, F.H.H. (1992). What parents know about preparing for college and how it 
affects their children’s academic performance: Parents’ information in the college choice decision . 
Unpublished doctoral dissertation. University of California at Berkeley. 

Owings, J.A. et al. (1995). Making the cut: Who meets highly selective college entrance 
criteria? Statistics in brief . NCES 95-732. Washington, DC: NCES. 



* The data elements are not currently available but are likely to be available in the next followup. 
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Steelman, L.C. & Powell, B. (1993). Doing the right thing: Race and parental locus of 
responsibility for funding college. Sociology of Education. 66 (4). 233-244. 



Issue: College entry rates: differences across student groups and institution types 

To inform policymaking, statistics on college entry rates are needed. Policymakers need to know 
the differences of entry rates across student demographic groups, higher education institution 
types, and geographic areas, as well as changes in the rates over time. Comparative analysis of 
the entry rates may help decisionmaking on resource allocation and procedural improvement 
related to higher education. 

The upcoming NELS:88 data will provide college entry rates for such analysis, and the data will 
be comparable with HS&B and NLS-72 data. 



2. Progression. Persistence, and Completion 

Issue: College education experience: academic work and school activities 

To learn about the processes of higher education, research should examine college students’ 
experience in both intellectual and social development. The academic experience includes major 
field of studies and possible changes, financial support, courses taken and credits earned by field, 
and academic achievement. Social experience includes extra-curricular activities, work experience, 
religion or community involvement, and housing arrangements. Student background and 
institutional characteristics can be related to different college experiences. 

The upcoming NELS.88 followup will provide data on college academic work (with transcripts 
data) and participation in campus activities.* 



Issue: Completion or withdrawal at the postsecondary level 

The issue of completion or dropping out at the postsecondary level can be linked to aspirations 
and high school experiences. Factors such as high school academic achievement (with high school 
transcripts), foreign language course taking, and advanced curriculum may contribute to college 
attendance, choice of college major, college completion, and post-college career development. 
While study on college completion awaits data from the fourth followup (in 1998 after cohort 
members supposedly complete college), other postsecondary education issues can be addressed 
with the third followup data. 

Caution: The validity of student reported high school experiences (engagement, course taking, 
aspirations) need to be checked against transcripts or the teacher file. 



* The data elements are not currently available but are likely to be available in the next followup. 
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Issue: Nontraditional (especially female) students in postsecondary education 

Many women go back to college after a career or family; retirees or mid-career employees pursue 
college or graduate education. They are different from traditional college students in background 
and experience. Many factors may cause the non-traditional cycle in higher education and in the 
labor force: for example, part-time versus full-time work (or college attendance), multiple 

campus attendance (within a state or cross-states); and changing social status and roles in relation 
to such nontraditional education. What are the motivations, supports, and outcomes of 
nontraditional college education? What are the institutional factors facilitating nontraditional 
college attendance? Does counseling help people to make decisions on such matters as going back 
to college, changing majors, and switching careers? The NELS.88 sample with the high school 
cohort is not designed to cover the nontraditional student population; however, BPS does provide 
information on these issues (e.g., college attendance, academic major, program taking, and 
working status). Other sources, such as B&B, the Recent College Graduates survey (RCG), and 
the Bureau of the Census household survey should be considered for such a study. 



3. Institutional Studies 

Issue: Financial aid provisions, enrollment, withdrawal and transfer, and program 
length: higher education institutions 

From the institutional perspective, higher education processes can be examined based on statistics 
aggregated from student surveys. Financial aid provided, student enrollment, withdrawal and 
transfer rates, selection and changes of academic fields, and the number of years taken to 
complete programs, are examples of key statistics for institutional research. Comparisons of such 
data across sectors, regions, academic disciplines/programs, and types of higher education 
institutions can help us learn about the operation of the higher education enterprise. Such studies 
may also offer implications for understanding changes in labor supply and demand. 

TheNELS:88 fourth followup will gather information about respondents’ college experience that 
can be used for institutional studies.* Additional data about higher education institutions are 
available from 1PEDS. 



Issue: Community colleges ’ growing and changing role in education 

Community colleges are growing into an important and increasingly diverse component in the 
nation’s postsecondary education system. It is necessary for policymaking purposes to know the 
multiple roles played by community colleges, including academic/vocational/technical training, 
adult and continuing education, service to underrepresented groups and disadvantaged areas, and 
local cultural and developmental activities. Institutional characteristics should be examined to see 
how they are associated with high rates of student retention or transference to four-year programs 
and high enrollment of disadvantaged groups. 



* The data elements are not currently available but are likely to be available in the next followup. 
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NELS:88 and the NCES higher education surveys (IPEDS, B&B, RCG, and BPS) contain 
institutional data. Statistics can be generalized to the national population from the representative 
institutional samples (except that from NELS:88, which did not sample institutions after the base 
year). 



4. Attainment of Professional and Graduate Education 

Issue: Choice of and entry into professional and graduate education programs 

Recent years have witnessed a significant increase of women and minority enrollment in 
professional and graduate education programs. The distribution of women and minority students 
across major academic fields (e.g., science/engineering, social sciences/humanities, medicine, law, 
and business administration), however, is not even. It is interesting to study gender- or race- 
differentiated enrollment of graduate programs by relating advanced program attendance to 
secondary and undergraduate college experience. A host of important questions can be asked: 
What motivates students to go into advanced programs? What is the impact of high school 
coursework on the choice of specific professional/graduate programs? When and how did the 
gender/race differences in academic field selection emerge? Why do some students change their 
fields between undergraduate and graduate programs? What are the financial sources for advanced 
studies and how does financial support influence program choice and completion? 

NELS:88 will collect data relevant to these questions in its fourth followup in 1998. The advantage 
of NELS:88 data (relative to the NCES higher education surveys) is its long range of data 
collection from middle school up to postgraduate education. In-depth studies can be formed to 
address issues related to professional and graduate education. 



Issue: Aspiration, choice, and completion of advanced programs in science and 

engineering 

It has been documented that, in the past decades, students’ aspiration for and performance in 
science/engineering learning is declining. To maintain the nation’s strong foothold in the global 
technological and economic competition, graduate education in science and engineering has a vital 
role to play. Studying the experience of students who are motivated to and succeed in advanced 
science/engineering training may shed light on policymaking and program planning aimed to 
improve high level training in these fields. Relevant to the issue of equity, study of graduate 
experience in science and technology training among traditionally disadvantaged groups is also 
needed. It can inform decisionmaking regarding resources and incentives for encouraging 
science/engineering studies among disadvantaged groups. 

Again, with its rich data on student background, family and community, earlier stages of schooling, 
and college education, plus the to-be-collected data on graduate training, NELS:88 will be a 
promising source for research on graduate education and the career paths of scientists and 
engineers. 



46 N ELS: 88 Research Framework and Issues 



Motivation, Aspiration, Self-Concept, and Other Psychosocial Issues 



D. Motivation, Aspiration, Self-Concept, and Other Psychosocial Issues 

Social psychological study of student motivation, aspiration, self-concept, attitudes, and perceptions 
has accumulated a large literature. These issues face researchers at both secondary and 
postsecondary levels. 



1 . Educational and Occupational Aspiration 

Issue: Minority students ’ aspirations and the reality of their educational and occupational 

opportunities 

Student aspirations are an important determinant of postsecondary educational attainment and 
occupational success for under-represented groups. Among minority students, aspirations can be 
examined against the reality (their academic performance and lifestyle, access to information about 
opportunities for higher education, and their parents’ expectations and planning). How do students’ 
"realistic" aspirations affect postsecondary outcomes? Some incongruency is often found among 
African American students. They aim high but have little access to information and opportunities 
and are poorly prepared in academic work. 

Relevant literature 



Braddock, J.H.ll et al. (1991). Bouncing back: Sports and academic resilience among 
African-American males. Education and Urban Society. 24 (1). 113-31. 

Catsambis, S., Mulkey, L., Crain, R. & Ascevedo, M. (1994). Gender and middle-school 
tracking effects on student attitudes, engagement, aspirations and locus of control . Paper presented 
at the meeting of the American Sociological Association, Los Angeles, CA, August 1994. 

Conway, K.P. (1994). Parent involvement, gender and race/ethnicitv: The relationship of 
student and self-concept . Unpublished doctoral dissertation. Hofstra University. 

Green, P., Baldridge, J.G., Zahs, D. & Ingels, S.J. (1993). High school seniors look to the 
future. 1972 and 1992. Statistics in brief . NCES 93-473. Washington, DC: NCES. 

Green, P.J., Dugoni, B.L., Ingels, S.J. & Cambum, E. (1992). A Profile of the American 
high school senior in 1992 . NCES 94-384. Washington, DC: NCES. 

Ingels, S.J., Plank, S.B., Schneider, B. & Scott, L.A. (1994). A Profile of the American 
high school sophomore in 1990 . Washington, DC: NCES. 

Smith-Maddox, R.P. (1994) African American eighth graders: Factors affecting their 
educational and occupational aspirations . Unpublished doctoral dissertation. Brandeis University. 

Solorzano, D.G. (1992). An Exploratory analysis of the effects of race, class, and gender 
on student and parent mobility aspirations. Journal of Negro Education. 61 (1), 30-44. 



Issue: Science/math learning aspiration and achievement under peer influence in different 
school/community contexts 

This issue considers how children with the same interests influence each other in science and math 
learning and how school and family environments foster positive peer influences on science and 
math learning. Specifically, what is the impact of peer influence relative to the impact of school 
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programs and classroom instruction? How does this kind of peer influence differ across gender and 
race/ethnicity groups? How does the peer influence interact with school programs and school 
climate in affecting math/science learning? What are the effective approaches by which school and 
teachers can mediate the effect of peer culture? What is the role of family and community in 
encouraging students to study math and science? Some integrated theoretical models (e.g., 
educational productivity model) are available to guide systematic research on both the cognitive 
and psychosocial effects of learning. 

NELS:88 provides relevant data for such studies, including peer values regarding academic learning 
in general and specific subjects; communication with peers, teachers, parents, and other significant 
others; and frequency and time spent with friends. Relating these data with data on students’ 
likings, aspirations, and learning processes on math and science, as well as the influences of 
parents, teachers, and school programs, it is feasible to operationalize research on peer influence. 

Relevant literature 



Catsambis, S. (1994) The path to math: Gender and racial-ethnic differences in mathematics 
participation from middle school to high school. Sociology of Education. 67 (3), 199-215. 

Horn, L. & Hafher, A. (1992). A Profile of American eighth-grade mathematics and science 
instruction. NELS:88. Statistical analysis report . NCES 92-486. Washington, DC: NCES. 

Peng, S.S. & Hill, S.T. (1994). Characteristics and educational experiences of high- 
achieving minority secondary students in science and mathematics. Journal of Women and 
Minorities in Science and Engineering. 1. 89-104. 

Peng, S.S., Wright, D. & Hill, S.T. (1995). Understanding racial-ethnic differences in 
secondary school science and mathematics achievement. Research and development report . NCES 
95-710. Washington, DC: NCES. 

Pittman, S.G. (1992). The Effect of educational productivity factors on eighth-grade interest 
and achievement in science . Unpublished doctoral dissertation. University of Illinois at Chicago. 

Snow, R.E. & Ennis, M.M. (1994). Correlates of high mathematical ability in a national 
sample of eighth graders. In C.P. Benbow& D. Lubinski (Eds.), From Psychometrics to giftedness . 
Baltimore, MD: Johns Hopkins University Press. 



2. Self-Concept. Locus-of-Control. and Social Behavior 

Issue: Student self-concept, self-esteem, and locus-of-control 

Theoretically, self-esteem and related social psychological concepts are useful in explaining 
academic achievement and educational attainment. Many questions, however, remain unanswered 
regarding how self-concept evolves and how the changes affect learning and behavior. The process 
in which children modify their conception of self over the years of schooling is not clear. Research 
needs to pinpoint important factors that contribute to development of positive self-concept. 
Racial/ethnic identity, school environment, peer recognition, and teachers’ assessment are 
conventional predictors of self-esteem. Academic performance and self-esteem, on the other hand, 
may be related to one another in complicated ways. Black children, for example, were found to 
have high self-esteem that is relatively independent of academic achievement. 
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NELS.88 data cover major psychosocial constructs relevant to self (e.g., self-esteem, self-direction, 
self-concept, self-confidence, fatalism, locus-of-control) with consistent measures of these concepts 
in base-year and each followup. With data on student background and school experience, NELS:88 
makes it possible to study complicated relationships between self-concept and education experience 
and outcomes. 

Relevant literature 



Collins, M.A. (1992). School social context, self-esteem, and locus-of-control among white, 
black, and Hispanic youth . Unpublished doctoral dissertation. University of Maryland. 

Conway, K.P. (1994). Parent involvement, gender and race/ethnicitv: The relationship of 
student and self-concept . Unpublished doctoral dissertation. Hofstra University. 

Driesler, K.J. (1993). Self-concept, locus-of-control and the home environment as predictors 
of reading comprehension in eighth-grade students . Unpublished doctoral dissertation. University 
of Tennessee. 

Huang, G.G. (1992). Self-reported biliteracv and self-esteem: A study of Mexican American 
eighth graders . Charleston, WV: Appalachia Educational Laboratory. Education Resource and 
Information Center Document No. ED 356937. 

Marsh, H.W. (1994). Using the National Longitudinal Study of 1988 to evaluate theoretical 
models of self-concept: The self-description questionnaire. Journal of Educational Psychology. 
86(3), 439-56. 



Issue: Substance use among high school and college students 

Recent research indicates that substance use among the younger population, particularly marijuana 
use among high school and college students has been increasing since 1990, after a decline in the 
1980s. The possible new rising trend of drug abuse among adolescent and young adults is alerting 
policymakers and practitioners to come up with new solutions. Understanding the causes of drug 
use, thus, is critical for policymaking and program planning. 

NELS:88 provides standardized measures of major classes of substance use with lifetime, annual, 
and monthly incidence data. The data set may be helpful for explaining, rather than describing, the 
behavior of substance use. Because of the large time span of its longitudinal design and dense 
information about school, family, and peer groups, NELS:88 is good for tracing the drug use career 
(initiation, gatewaying, changing pattern of use, and cessation); evaluating the impact of family, 
community, and school experience on the behavior; and examining the long-term consequences of 
drug use. 

Relevant literature 



Dewalt, M.W. (1992). The Relationship between youth group participation and use of 
tobacco and drugs . Paper presented at the meeting of the American Educational Research 
Association, San Francisco, C A, April 1 992. Education Resource and Information Center Document 
No. ED 350375. 

. (1993). Using youth group participation variables and smoking to predict drug use 

in a national sample . Paper presented at the meeting of the American Educational Research 
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Association, Atlanta, April 1993. Education Resource and Information Center Document No. ED 
363558. 



Issue: Students ’ psychoemotional problems and well-being 

It has been widely accepted that adolescence is a development stage difficult to the individual 
because of rapid physical and psychosocial growth, which is often complicated by changes in 
family, school, and the society. High school students experience various adjustment problems 
taking place both in themselves and the environment. Research should monitor the psychoemotional 
well-being of youth and inform practitioners and policymakers so they can come up with 
responsive services. 

The NELS:88 longitudinal data on psychosocial and emotional development and relevant services 
received offer an opportunity for such research. 

Relevant literature 



Lee, V.E., Burkam, D.T., Zimiles, H. & Ladewski, B. (1994). Family structure and its 
effect on behavioral and emotional problems in young adolescents. Journal of Research on 
Adolescence. 4 (3), 405-437. 

Mulkeen, P. (1992). Adolescent activities in school and the community: Patterns of 
participation . Unpublished doctoral dissertation. Pennsylvania State University. 

Sayler, M.F. & Brookshire, W.K. (1993). Social, emotional, and behavioral adjustment of 
accelerated students, students in gifted classes, and regular students in eighth grade. Gifted Child 
Quarterly. 37 (4). 150-154. 

Weishew, N.L. & Peng, S.S. (1993). Variables predicting students’ problem behaviors. 
Journal of Educational Research. 87 (1). 5-17. 

E. Methodological Research 

Methodological/technical matters are always intertwined with substantive issues. We limit the 
methodological presentation to issues that are primarily methodological concerns and that can be 
addressed with the currently available data. Methodological problems that concern program 
improvement that cannot be attempted with the available data will be presented elsewhere. 



1 . Data Quality and Measurement 
Issue: Data validating 

Data validating is a pressing issue for improving NCES data quality. Research has found problems 
in data validity, problems which were caused by irregular responses or inappropriate instrument 
design. African American male students tend to provide irregular responses to surveys to a much 
greater extent than other students. Social desirability can undermine data validity. Different or 
conflicting views on school issues are held by students, teachers, and parents; the perceptions of 
low ability students and high ability students differ. The meanings of such responses should be 
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sorted out. Such efforts can help improve survey design within relevant theoretical or policymaking 
frameworks. 

With its comprehensive data contents and multiple data sources (students, teachers, parents, and 
administrator), NELS:88 should be subject to data validation. For example, 99 percent of the 
teachers in the survey answered that they stressed critical thinking skills, but we know that is not 
the case. Checking against student responses to questions on classroom process may help figure 
out the instructional reality. A "convergent" approach, in which the same information is collected 
from different informants (the student, his/her teacher and parents, and the school administrator), 
would enable researchers to examine data validity. Such a data validating process can also 
generate information to determine the best possible sources and best possible questionnaire items 
on specific issues for future survey design. 

Relevant literature 



Dowd, K.L. et al. (1991). The NELS:88 second followup field test report . Chicago, IL: 
National Opinion Research Center. 

Greene, H. et al. (1989). National Education Longitudinal Study of 1988 (NELS:88) first 
followup field test report . Chicago, IL: National Opinion Research Center. 

Hamilton, L.S., Nussbaum, M., Kupermintz, H., Kerkhoven, J.I.M. & Snow, R.E. (1994). 
Enhancing the validity and usefulness of large-scale educational assessments II: NELS:88 science 
achievement . Stanford, CA: Center for Research on the Context of Secondary Teaching, Stanford 
University. 

Ingels, S.J. et al. (1987). National Education Longitudinal Study of 1988. Field test report . 
Chicago, IL: National Opinion Research Center. 

Kaufman, P. & Rasinski, K.A. (1991). Quality of the responses of eighth-grade students 
in NELS: 88-National Education Longitudinal Study of 1988 . Technical report. NCES 91-487. 
Washington, DC: NCES. 

Kupermintz, H., Ennis, M.M., Hamilton, L.S., Talbert, J.E. & Snow, R.E. (1994). 
Enhancing the validity and usefulness of large-scale educational assessment I: NELS: 88 
mathematics achievement . Stanford, CA: Center for Research on the Context of Secondary 
Teaching, Stanford University. 

McGrew, K.S. et al. (1994). Matching information in national data collection programs 
to a model of school completion outcomes and indicators. Technical Report 7 . Minnesota 
University, Minneapolis Coll, of Education: National Association of State Directors of Special 
Education. Alexandria, VA. National Center on Educational Outcomes. Minneapolis, MN: Saint 
Cloud State University, MN. 

Pollack, J.M. (1990). Some issues in free response testing . Princeton, NJ: Educational 
Testing Service. 

Rasinski, K.A. (1991). How seriously should we take children’s reports about the 
seriousness of problems in their schools? A psychological analysis of the validity of children’s 
survey responses . Paper presented at the meeting of the American Association for Public Opinion 
Research, St. Petersburg, FL, May 1989. 
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Issue: Test indicator scales with the multilevel structural equation technique 

The indicator systems can be studied with multilevel structural equation modeling (with packages 
such as LISREL or PRE LIS). This approach, combining the strength of multilevel regression and 
path analysis techniques, promises to model student achievement by taking into account the fact 
that differences in policies and practices occur at the school and classroom levels. The model can 
be used to validate the indicators and to study the effects of changes in policy-related variables 
on achievement. Ongoing research has explored science achievement modeling and the related 
indicator system. 

These studies examine the existing items in NELS:88 achievement scales to see how they 
differentially reflect the underlying concepts and how they are related to policy-relevant variables. 
Such studies can help refine the indicator systems by making them more sensitive to policy 
variation while increasing survey efficiency. Besides cognitive tests, other indicator systems, such 
as those for school climate, student self-esteem, parental involvement, and academic motivation 
can be considered. NELS:88, HS&B, NLS-72 contain such multi-item scales. 

PRE LIS or other sophisticated software is needed to examine both construct validity and 
reliability. PRE LIS does confirmative factoring based on some theoretical constructs. Examining 
the factor correlation matrix resulting from LISREL can help eliminate some trivial items that 
load little on the conceptual construct. The approach is a solid way to test a hypothesis since 
some theoretic presupposition was imposed upon the data. 

Caution: The small number of students nested in each classroom (which in turn is nested in each 
school) in the NELS:88 sampling design makes it impossible to deal with between-classroom 
differences. Another problem is related to the theoretical basis of such modeling. We assume a 
common factor structure across all schools in considering school policies and school climate. But, 
in reality, the structure can vary across schools. There may be different underlying factors for the 
notion of school climate (not just differences in the magnitude of factor loadings of items) 
depending on different school types (e.g., private versus public, disadvantaged versus 
advantaged). To allow the school level variables to vary and to investigate the implications of 
such school-level variance, we may want to do separate multilevel factor analysis with different 
types (groups) of schools. This way we can see different climate factor structures in different 
types of schools. Future research should consider this important issue. 



Issue: School climate measures and other multiple-item indicator scales 

The existing school climate measures have little predictive validity (i.e., they do not predict 
outcomes well). It is not that the idea of school climate is unimportant; rather, it is that measuring 
school climate is a problem. The composite score is not useful because different people perceive 
the question differently. What is the difference really meant by respondents when they say 
"disagree" in contrast to "agree"? A study shows that school climate variance occurs mostly 
within school; i.e., different views are held by teachers within the same school, not across 
schools. It is important to have both accurate measurement and an appropriate analytic approach 
at the school and individual levels. 
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In addition to school climate measures, NELS:88 also contains other indicator scales with 
multiple items, such as that of self-concept and opportunity to learn, which also need study. 
These scales can be examined with NELS:88 data. The results can be compared with that from 
other data sets that provide indicators for the same constructs (e.g., NAEP, SASS). 

Relevant literature 



Bobbitt, S.A. (1993). Using opportunity to learn items in elementary and secondary 
national surveys . Paper presented at the meeting of the American Educational Research 
Association, Atlanta, April 1993. Education Resource and Information Center Document No. ED 
363667. 

Marsh, H.W. (1994). Using the National Longitudinal Study of 1988 to evaluate 
theoretical models of self-concept: The self-description questionnaire. Journal of Educational 
Psychology. 86 (3). 439-456. 



Issue: Missing data, nonresponse analysis, and imputation procedures 

Missing data caused by nonresponse are a thorny problem facing survey researchers. Nonresponse 
bias, conceived as a function of the nonresponse rates and the difference between respondents and 
nonrespondents, can cause misleading estimates if not handled appropriately. Unit nonresponse 
and item nonresponse, with different causes and impact on the survey results, are not consistently 
defined nor adequately examined in NCES projects. Nonresponse analysis and imputation 
procedures can help improve analysis of the existing data. Further, information about 
nonrespondents and the pattern wherein nonresponses happen can shed light on survey program 
improvement. 

NELS:88’s weighting procedures include adjustment for unit nonresponse. Item nonresponse, 
however, has not been compensated (with some exceptions). Studies are needed to systematically 
examine the nonresponse patterns and the bias to data analysis. 

To examine unit nonresponses, one can use descriptive analysis of the available information about 
nonrespondents and respondents, linking the differences to key variables of respondent 
background if possible. For item nonresponse research, descriptive analysis of items can be done 
to see how item nonresponse relates to respondent characteristics, the content of the item (e.g., 
sensitivity and difficulty), and the questionnaire structure (e.g., sequencing of items, skip patterns, 
and the number of items in the questionnaire). Descriptive analysis of item nonresponse 
concerning specific substantive issues can be done with a similar approach. 

Relevant literature 



Dowd, K.L. et al. (1991). The NELS:88 second followup field test report . Chicago, IL: 
National Opinion Research Center. 

Ingels, S.J. et al. (1994) Second Followup Student User’s Manual (section III-3.4; section 
V-5.1 to 5.3; section VI-6.1; appendix P) "Quality of responses of eighth graders in NELS:88". 
Washington, DC: NCES. 
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Ingels, S.J. et al. (1989). School, individual, and item nonresponse in the National 
Education Longitudinal Study of 1988 (TsfELS:88) Base-year survey . Paper presented at the annual 
meeting of the American Educational Research Association, San Francisco, CA, March 1989. 

Saba, M., Fink, S., Chang, M. & Peng, S. (in press). Documentation of nonresponse in 
NCES surveys . Arlington, VA: Synectics (forthcoming). 

Witta, E.L. (1992). Seven methods of handling missing data using samples from a national 
data base . Unpublished doctoral dissertation. Virginia Polytechnic Institute and State University. 



Issue: Psychometric study of achievement gains in NELS.88 and HS&B 

Psychometric research on comparability of achievement measures of different surveys is needed 
to set the ground for cross-cohort studies. Meaningful intercohort comparison is possible only 
when the measures for different cohorts are comparable. Analysis of data on sophomore to senior 
year gains with NELS.88 and HS&B data will contribute to this research. 

NELS.88 includes a number of mathematics test items identical to those in HS&B. These items 
are sufficient for a common item equating study that may develop the same scale for the HS&B 
math test as that in NELS:88. Researchers could fix the common items from the two tests at the 
NELS:88 item parameter values and then put the remaining HS&B items on that scale. A 
multiple group Bayesian estimation procedure may be used for such a study. 

Relevant Literature 



Hamilton, L.S. et al. (1994). Enhancing the validity and usefulness of large-scale 
educational assessments: II. NELS:88 Science Achievement . Paper presented at the Annual 
Meeting of American Educational Research Association, New Orleans, LA, April 1994. Center 
for Research on the Context of Secondary School Teaching. 

Kupermintz, H. et al. (1994). Enhancing the validity and usefulness of large-scale 
educational assessments; NELS:88 mathematics achievement . Paper presented at the Annual 
Meeting of the American Educational Research Association, New Orleans, LA, April 1994. 
Center for Research on the Context of Secondary School Teaching. 



2. Research on Sampling Problems 
Issue: Development of school sample weights 

The NELS.88 school sample is not nationally representative (except for the base year). For 
example, urban schools and disadvantaged schools with special groups (e.g., Hispanic students) 
were probably undersampled. This presents difficulties for studying school factors, which are 
essential in institutional and policy research. Investigation is needed to develop school weights 
to make NELS:88 school data representative of the nationwide high school population. Students’ 
transition from middle school to high school may complicate the sample. Some schools in the 
sample have very few students after the base year. Study is needed to tackle the difficulties 
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arising from the small size of within-school student samples. Post-stratification adjustment seems 
to be an option for such a study. 

Relevant literature 



Spencer, B.D. & Foran, W. (1991). Sampling probabilities for aggregations, with 
application to NELS:88 and other educational longitudinal surveys. Journal of Educational 
Statistics. 16 (1), 21-33. 



Issue: Excluded students from N ELS: 88 sample and related problems 

Some students were excluded from the NELS:88 sample because their disabilities or because 
language barriers made it infeasible to survey them with questionnaires and cognitive tests. Such 
exclusion has implications for sample representativeness, national estimation, and interpretation 
in policy analysis. The excluded students represent groups that face high risk of school failure 
and that need special services. It is necessary to assess the impact of the exclusion on the survey 
results in order to draw valid conclusions on pertinent policy issues (e.g., special education, LEP 
programs, and dropout). For example, on variables relevant to policy issues, descriptive analysis 
can be used to compare excluded students (the base-year ineligible students and the two waves 
of freshened students) and sampled students. Interpretative analysis with state or local data 
sources in examining the results from NELS:88 may help verify the bias resulting from the 
exclusion. 

Relevant literature 



Ingels, S.J. et al. (1994). Second followup student user’s manual (section III-3.4.1 "Bias 
caused by undercoverage of special populations." Washington, DC: NCES. 

Ingels, S. J. (1991). The problem of excluded baseline students in a school -based 
longitudinal study: Correcting national dropout estimates and accommodating eligibility change 
over time . Paper presented at the meeting of the American Educational Research Association, 
Chicago, IL: April 1991. Education Resource and Information Center Document No. ED 334273. 

Ingels, S.J. & Scott, L.A. (1993). Strategies for including all students in national and state 
assessments: Lessons from a national longitudinal study . Paper presented at the National 
Conference on Large-scale Assessment of the Council of Chief State School Officers. 
Albuquerque, NM, June 1993. 

Ingels, S.J. & Scott, L.A. (1993). Exclusion of students with barriers to participation in 
NELS:88 — baseline excluded students two and four years later . Paper presented at the meeting 
of the American Educational Research Association, Atlanta, GA, April 1993. 

Ingels, S.J. & Owings, J.A. (1994). Methodological issues encountered in following a 
cohort of eighth graders . Education Resource and Information Center Document No. ED 350375. 

McGrew, K.S., Martha, L.T. & Spiegel, A.N. (1993). An investigation of the exclusion 
of students with disabilities in national data collection programs. Educational evaluation and 
policy analysis. 15 (31. 339-352. 
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National Center on Educational Outcomes. (1992) Including students with disabilities in 
national and state data collection programs . Minneapolis, MN: Author. Educational Resource and 
Information Center Document No. ED 347769. 



3. Analytic Techniques 

Issue: Level of methodological sophistication necessary for valid results 

What level of analytic sophistication is necessary for generating good data? What is the essential 
way to analyze data? This issue concerns the cost-efficiency of data analysis and has implications 
for increasing data utilization. 

NELS:88 affords a laboratory to test the essence of sophistication of analytic techniques. 
Comparing results from analyses with different levels of sophistication but which cover the same 
data may help address the issues. For example, contrasting the results from LISREL with the SAS 
maximum likelihood estimate procedure (MLE) may be helpful for determining more efficient 
procedures in data analysis. Or, one can compare results from hierarchical regression on student 
data nested in schools with that from conventional multiple regression (with sampling bias 
correction procedures) to identify what kind of differences occur in the results. Factors (e.g., data 
measurement, scaling, range) that alter sensitivity of the outcomes to analytic techniques can be 
considered. 



Issue: Using sampling weights in multilevel analysis and structural equation modeling 

Questions arise as to how to apply sampling weights to complicated analysis with multilevel 
regression analysis. In addition, the problem of normality assumption with multistage sampling 
makes the LISREL model questionable. Estimates of variance with a multistage sampling design 
require sampling bias adjustment. NELS:88 data can be used for such exploration, for example, 
using PRE LIS to test the difference in results from procedures with sampling error correction 
versus procedures without such correction. 

Issue : Application of Hierarchical Linear Modeling (HLM) 

HLM is increasingly frequently used in research with NELS:88 data. Problems or complications 
involved in using the procedure (e.g., centered score versus raw score for predictor variables; 
instability of the nesting of students in schools) should be studied and the results documented. 
Further, the ways to present HLM results in the literature are not consistent, and some do not 
make sense. Efforts are needed to develop better ways to present HLM results (e.g., fixed effects 
and random effects at two or more levels) to make the results more informative. For example, 
it is sensible to present more "predictions" — not just goodness of fit — and to interpret diagnostics 
in order to attain validation of the model. Confidence interval estimates should be presented in 
addition to effect size (coefficients). Again, the NELS:88 design and data make it interesting to 
conduct such studies. 
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Relevant literature 



Burton, B. (1993). Some observations on the effect of centering on the results obtained 
from hierarchical linear modeling . Paper presented at the meeting of the American Educational 
Research Association, Atlanta, April 1993. Education Resource and Information Center Document 
No. ED 363656. 

Raudenbush, S. W. (1993). A Cross random effects model for unbalanced data with 
applications in cross-sectional and longitudinal research. Journal of Educational Statistics. 18 (4), 
321-349. 



Issue: Design effects and their applications 

The use of estimated design effects is still an option for handling sampling bias in multistage 
sample data. It supplements the multilevel regression technique (if the nested subsample size is 
too small) and the formal sampling bias correction (e.g., if software packages such as SUDAAN 
are not available). However, study is needed to clarify the conditions under which using design 
effects for computing variance estimates and hypothesis testing can produce reasonably good 
results, as compared with other approaches to correcting multistage sampling bias. 



Issue: Prediction models of academic performance 

Prediction models of performance may be helpful for both advancing analytic techniques and 
learning about education policies. If we have test scores for the same student at the different 
grade levels (e.g., grades 8-12), we can predict 12th grade scores based on the initial scores and 
a growth projectory within the time frame, and then identify the difference between the predicted 
12th grade score and the actual 12th grade score. Though the predicted model may be a flat 
growth curve (which is not desirable), one can manipulate ("switch" on or off) certain variables 
either at school level (probably more policy-relevant) or individual level to see how the flat curve 
can be changed into a slope. Latent variables in the model are useful for such manipulation. The 
results can identify factors that are important in improving academic achievement growth. Such 
studies are underway [by the National Opinion Research Center (NORC) and some doctoral 
dissertations work with the growth curve technique] and the results probably will be available 
soon. 

This approach is consistent with a generic way of looking at the data and policy issues: if a 
correlation is found, then researchers can go further to check at the experimental level to see how 
it works to improve outcomes. 



Issue: Methods for intercohort analysis 

Comparative analysis with longitudinal survey data collected from different cohorts across 
decades is highly valuable for policy studies. Major historical changes in education legislation and 
policies can be examined in connection to learning, behavior, and academic performance among 
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students of different cohorts. The approach can be used for different comparisons. For example, 
at the student level, time-lag, fixed time, and longitudinal comparative analysis can be performed. 
The analysis involves technical difficulties such as combining data sets, handling different 
attrition rates, developing analytic strategies that appropriately compare statistics resulting from 
different surveys; establishing comparability of measurements used in different surveys, and 
interpreting subtle inconsistency of seemingly similar items across data sets. 

Methodological research and technical support is needed to develop sound strategies for 
intercohort analysis. NELS:88, HS&B, and NLS-72, with data that cover three decades of 
education changes in the U.S., offer abundant opportunities for exploring the issue. Further, one 
can compare statistical results from intercohort analysis to other information sources about 
historical changes in American education, including interpretive analysis — as in contrast to 
statistical analysis — of cross generation changes in broader literature of social sciences. 

Relevant literature 



Ingels, S.J. et al. (1994). Second followup student user’s manual. (Appendices D and E). 
Washington, DC: NCES. 

Drazen, S. (1992). Student achievement and family and community poverty: twenty years 
of education reform . Paper presented at the meeting of the Eastern Psychological Association, 
Boston, MA, April 1992. 

Green, P., Baldridge, J.G., Zahs, D. & Ingels, S.J. (1993). High school seniors look to the 
future, 1972 and 1992. Statistics in brief . NCES 93-473. Washington, DC: NCES. 

Green, P.J., Dugoni, B.L., Ingels, S.J. & Camburn, E. (1992). A Profile of the American 
high school senior in 1992 . NCES 94-384. Washington, DC: NCES. 

Green, P.J., Dugoni, B.L. & Ingels, S.J. (1992). Trends among high school seniors 1972- 
1992 . NCES 94-380. Washington, DC: NCES. 

Ingels, S.J. & Baldridge, J.B. (1994). Conducting trend analyses: NLS-72. HS&B, and 
NELS:88 seniors . Washington, DC: NCES. 

Ingels, S.J. & Dowd, K.L. (1994). Conducting trend analyses: NLS-72. HS&B. and 
NELS:88 sophomore cohort dropouts . Washington, DC: NCES. 

Ingels, S.J. & Taylor, J.R. (1994). Conducting cross-cohort comparisons using HS&B. 
NAEP. and NELS:88 academic transcript data . Washington DC: NCES. 

McMillen, M.M., Hausken, E.G., Kaufman, P., Ingels, S.J., Dowd, K.L., Frankel, M.R. 
& Qian, J. (1993). Dropping out of school: 1982 and 1992. Issue brief . NCES 93-901. 
Washington, DC: NCES. 

Rasinski, K.A., Ingels, S.J., Rock, D.A. & Pollack, J.M. (1993). America’s high school 
sophomores: A ten year comparison. 1980-1990 . NCES 92-087. Washington, DC: NCES. 
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Base-year ineligible students - The NELS:88 Base-year Survey excluded students who were 
in the sample but could not complete the questionnaire due to language or physical/mental 
disabilities. Among this group, those who became eligible at the time of the first followup 
were located and surveyed by the NELS:88 Base-year Ineligible Study. 

Ceiling effect - The result of a cognitive test having insufficient numbers of the more 
difficult items. In a longitudinal study, ceiling effects in the followup testings can cause 
change scores to be artificially constrained for high ability examinees. To reduce such bias, 
we can match test item difficulty to a person’s ability level; this increases reliability at the 
extremes of the score distribution where it is most needed for studies of longitudinal change. 
The NELS:88 first and second followups used what was called "proficiency level tests" to 
minimize ceiling effects and floor effects (see also Floor effect). The multilevel tests are 
"adaptive" to the ability level of the students as they are assigned based on prior test 
performance. The results not only reduce the bias of the estimated score gains, but also allow 
more meaningful comparisons by linking score gains to learning and instruction. 

Centered score - A data transformation procedure used in hierarchical data analysis (e.g., 
with student- and school-level data); it typically involves subtracting the mean (either grand 
mean or school means) from the raw value of a student-level predictor variable. The 
procedure may make estimates more stable and intercepts more meaningful in interpretation. 

Cohort - A group of individuals who have a statistical characteristic in common, for example 
year of birth or grade in school or year of high school graduation. NELS:88 includes three 
overlapping but distinct nationally-representative grade cohorts: 1987-88 eighth graders, 
1989-90 high school sophomores, and 1991-92 high school seniors. 

Composite - A derived variable that is created as a function of one or more variables for 
which data was directly collected (e.g., socioeconomic status is developed from five parent 
variables in NELS:88). 

Cross-sectional analysis - An analysis that focuses on events and statuses at a single point in 
time. For example, a cross-sectional analysis examines educational attainment of students at a 
particular stage of schooling (e.g., eighth grade, tenth grade, or twelfth grade). NELS:88 is a 
longitudinal survey but with its sample freshening, three representative cross-sections are 
available for cross-sectional analysis (see also Freshening and Longitudinal analysis). 

Design effects - A measure of the efficiency of a sample. The design effect is the sampling 
variance of an estimate divided by the sampling variance of the estimate that would have 



* This glossary covers technical terms used in this publication. Information was largely drawn from the Glossary 
in the Second Followup: Student Component Data File User’s Manual (Ingels et al. 1994). Additional definitions 
of statistic terms were collected from other sources. 
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occurred if a sample of the same size of ultimate units had been selected using simple random 
sampling. 

ECB (electronic codebook) - A menu-driven system that allows users to perform functions 
such as the following: search a list of database variables by key words or variable 
names/labels; display percentages for each variable in the database; display question text for 
each variable in the database; select variables for analysis; generate SAS or SPSS program 
code/command statements for constructing a system file; and generate a codebook of the 
selected variables. An ECB with NELS:88 data is available on CD-ROM. 

Excluded students - See Base-year ineligible students. 

Factor analysis - A branch of multivariate analysis with applications chiefly in the social 
sciences. It resolves a set of variables (say p) linearly in terms of (usually) a much smaller 
number (m) of categories or "factors." This resolution is accomplished by the analysis of 
pairwise correlations among the variables. A satisfactory solution yields factors which contain 
all the essential information of the original set of variables and thus attains scientific 
parsimony or economy of description. One such model is mathematically expressed by 



i =Y, Aijfj+bisi +ciei(i =l,...p) 
j - 1 

where x : is the observed variate, fj is the j Ih factor, Sj a factor specific to the i th variate, and e : 
is an error variate, and a, b, and c’s are structural constants of the model. 

Flag - A distinctive special outcome of a variable (e.g., dropout in student status). 

Floor effect - The result of a cognitive test being too difficult for a large number of the 
examinees, causing the low ability examinees to receive chance scores on the first testing, and 
on subsequent testings if the test remains too difficult. Floor effects make the test unable to 
discriminate among low ability students or to detect their score gains over time. NELS.88 
used proficiency tests to reduce floor effects (see also Proficiency test). 

Freshening - A NELS:88 sampling procedure by which high school sophomores who were 
not in the eighth grade in the U.S. in 1988 were added in the first followup. This process was 
repeated in the second followup, adding high school seniors who were not in the eighth grade 
in the U.S. four years before, and not in the tenth grade in the U.S. two years before. This 
freshening ensured that the student sample would be representative of the 1992 senior class by 
allowing 1992 seniors who did not have a chance for selection into the base year (or the first 
followup) sample to have some probability of 1992 selection. 

Hierarchical Linear Model (HLM) - In the social sciences where data structures are often 
hierarchical in nature in the sense that variables describe individuals while individuals are 
grouped into larger units. And there are other variables describing these higher order units. 
Hierarchical linear models have been developed for analyzing such data using regression 
methods. This class of models takes the hierarchical structure into account and make it 
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possible to incorporate variables from all levels in the regression analysis. In an education 
system, students are members of classes, and classes are grouped within schools. HLM can be 
used to analyze such relationships such that, among other advantages, coefficients in a linear 
model of a process occurring at one level of a hierarchical system can be viewed as variables 
of interest that are functions of characteristics of units at another level. 

Imputation - A procedure whereby missing values in a questionnaire (due to item 
nonresponse) are filled in by a substitute value like the mean or obtained from a similar unit 
in the same survey (hot deck) or a previous survey (cold deck) or estimated from other 
variables (regression methods). 

Ineligible students - see Base-year ineligible students. 

Intercohort analysis - Analysis involving more than one cohort. For example, comparisons 
made between NELS.88, HS&B, and NLS-72. Different strategies can be used: (a) time-lag 
comparison, for example, comparing NELS:88 data as the third in a series of repeated cross- 
sections of twelfth graders, or comparing changes of the NELS:88 cohort in a time span of 
specified years with changes of HS&B and NLS-72 cohorts between the same time span; (b) 
fixed-time comparison, in which groups from different studies are compared to each other at 
different ages but at the same point in time (e.g., comparing the 1986 data of senior cohorts 
of NLS-72 and HS&B, at age of 32 and 24 respectively); and (c) longitudinal comparison by 
modeling the history of the age/grade cohorts. 

Longitudinal analysis - Also called panel design, in which similar measurements — of the 
same sample of individuals, institutions, or other unit of analysis — are taken at multiple time 
points. NELS:88 employs a longitudinal design that follows the same student over time, thus 
permitting longitudinal analysis of student-level change (see also Cross-sectional analysis). 

Multilevel analysis - see HLM (hierarchical linear model). 

Multilevel factor analysis - Use of factor analysis at different levels in a hierarchical 
situation (see factor analysis). 

Multilevel path analysis - Use of path analysis at different levels in a hierarchical situation 
(see path analysis). 

Multilevel structural equation - Use of structural equation models at different levels in a 
hierarchical situation (see structural equation models) 

Nonresponse bias - Error in estimation caused by (a) the proportion of surveyed persons 
who did not provide response to the questionnaire or an item and (b) the difference between 
respondents and nonrespondents in the population. 

Path analysis - Study and analysis of causal relations in sociological and statistical research 
whose purpose is to examine and test alternative hypotheses or alternative independent 
variables and to help establish causal connection and inference. In multiple regression, each 
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predictor variable has a direct effect on the response variable. Sometimes the predictor 
variable affects the response variable not only directly but also indirectly through one or more 
intervening variables. Path analysis is a technique for analyzing this kind causal analysis and 
builds on ordinary multiple regression. H. Blalock first examined path analysis in detail in his 
book Causal Inference in Nonexperimental Research (North Carolina: North Carolina Press, 
1961). 

Post-stratification adjustment - A weight adjustment that forces survey estimates to match 
independent population totals within selected poststrata (adjustment cells). 

Proficiency test - A multilevel test with differential difficulty levels matching examinees’ 
ability level determined by prior test performance. The results of such tests reduce ceiling and 
floor effects bias on score gains (see also Ceiling effect and Floor effect). 

Stratified sampling design - A sampling design which stratifies the population of interest 
into a number of mutually exclusive strata or subpopulations (usually homogeneous) and then 
draws simple random samples from each of the strata formed. 

Structural equation model - An expression which is usually employed to denote a 
mathematical relation among variables entering into the specification of a model and hence 
expressing its structure. In statistics and economics the variables are frequently stochastic in 
character, and the term has been used to differentiate between such relations and those of a 
functional kind in the ordinary mathematical sense. A distinction is sometimes made between 
forecast models and structural models. The former does not aim at describing the underlying 
phenomenon being studied but rather try to predict outcomes while the latter is more focused 
on describing the process that produces the phenomenon. In path analysis, a model without 
feedback loops and with uncorrelated errors (between equations and within equations) is 
called a recursive model. Structural equation model is a larger class that includes all recursive 
models as well as some nonrecursive models. 

Weights - A variable that is used to weight sample responses to population totals (e.g., the 
24,599 base-year students represent 3,000,000 eighth graders nation-wide). 
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Working with Composite Variables 

All NELS:88 student data files, Base-Year (BY), First Followup (FI), Second Followup (F2), 
and Third Followup (F3), include both basic demographic composites, such as gender, 
race/ethnicity, SES, and school level composites, such as school enrollment and school sector 
type (i.e., public, Catholic, nonsectarian private, and other religious schools). An 
understanding of which version of a composite variable to select and use is essential to 
constructing a good working data file (the first time around) and performing sound analyses. 
For example, should the race/ethnicity composite from the BY student data file or the FI 
student data file or the F2 student data file be selected? 

When constructing a working data file with composites, the researcher should follow two 
rules for selecting composites. First, with one exception, the most appropriate student file 
from which to select composites is the student file from the most recent round (and 
recent file) of NELS:88 in which such composites appear. While some variables, such as 
race/ethnicity and gender, should in theory remain constant for individuals over time, in 
practice they may change if new information which improves upon the old is collected. Thus, 
select composites once and select them from the student data file from the most recent round 
(and recent file) of NELS:88. 

There is one exception to the above rule. When the analysis population is the eighth grade 
cross-sectional cohort, SAMPLE SIZE=24,599, select composites from the BY student data 
file. In this situation the composites that appear on the BY student data file should be selected 
because they cover the full BY cross-sectional sample of 24,599, whereas, owing to student 
subsampling in the FI, the composites that appear on the FI and the F2 student data files 
cover only BY sample members retained in the study, and not all 24,599 originally surveyed 
eighth graders. 

REFERENCES 

Second Followup Student’s User’s Manual (section IV; appendix FI) and Base-year Student 
Component Data File User s Manual (section 4.4) 

Subgroup Analyses 

In addition to supporting multiple levels of analyses, NELS:88 is designed to support 
examination of specific policy-relevant subgroups. One such subgroup is dropouts. For the 
NELS:88 Base-year to First Followup longitudinal cohort, nearly 6.1 percent (does not include 
base-year ineligibles) had dropped out of school by the spring of 1990; and, for the First 
Followup sophomore longitudinal cohort, a similar 6.2 percent dropped out of school between 
the spring of 1990 and the spring of 1992. NELS:88 is also designed to produce estimates for 
specific racial-ethnic subgroups. Private school students, Hispanics and Asians were selected 
at a higher than normal rate in the base year and have been disproportionately retained in the 
study. 
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REFERENCES 

Second Followup Student Component Data File User’s Manual: Section 3.4.1 — Bias Caused 
by Undercoverage of Special Populations, Appendix H — NELS.88 Student Data Weights, 
Flags, and Composite Variables (F2), and Appendix D — Conducting Trend Analyses of 
HS&B and NELS.88 Dropouts 

Use of Design Weights 

Sample Design. The NELS:88 survey used a two-stage stratified, clustered sample design. 
NELS:88 students were selected in the base year using a two-stage probability sample with 
schools as the first-stage units and students within schools as the second-stage units. Certain 
groups (for example, Asian and Hispanic students, private school students) were 
disproportionately selected, to ensure a large enough analysis sample for these subgroups. A 
subsample of base year (1988) eighth graders was re-surveyed in 1990 and 1992. (In the 
subsampling process, dropouts were retained with certainty, while Hispanics, Asians, and 
American Indians were disproportionately retained.) The sample was "freshened" at both of 
these times. That is, some individuals were added to the sample who were 1990 sophomores 
or 1992 seniors but who were not eighth graders in the spring term of 1988 and thus had no 
chance of selection into the base year. Because of sample freshening, NELS:88 constitutes a 
nationally representative sample of spring term 1990 sophomores and spring term 1992 
seniors, as well as 1988 eighth graders. 

Weighting. Weighting is used in sample surveys to accomplish a number of objectives: 

• expand counts from sample data to full population levels; 

• adjust for differential selection probabilities; and 

• adjust for differential response rates. 

Because many populations of interest can be analyzed with NELS:88 data (base-year schools, 
1988 eighth graders two and four years later, 1990 sophomores, 1992 seniors, parents of 
students and dropouts in 1992, and so on), 12 separate "final" (or nonresponse-adjusted) 
weights have been produced. These weights appear on the data set — two for the base year, 
four for the first followup, and six for the second followup. Guidance on using the weights is 
provided below. 

Weights. Unweighted frequencies (and percents) reflect the NELS:88 sample; weighted 
frequencies (and percents) reflect the national or regional population estimate that is derived 
from the sample. For example, there are 1,032 schools represented in the NELS:88 base-year 
school file. This sample of schools represents 38,774 schools with eighth grades in the 1987- 
88 school year. The number of schools in the population that are represented by a given 
NELS:88 participating sample school is variable — specifically, it varies from a minimum of 
1.5 schools to a maximum of 387.3, with a mean of 37.5. Because of this variability in the 
weights, weighted and unweighted percentages will differ; true population estimates can only 
be produced through use of the appropriate weights. For example, if you want to examine the 
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proportion of American schools with eighth grades in 1 988 that offered an academic honor 
society available to eighth graders, you will find that unweighted and weighted proportions 


would differ 


in the following way: 






BYSC46A 


Academic Honor Society Available to Eighth Grader 




Unweighted 


Unweighted 


Weighted 




Frequency 


Percent 


Percent 


Yes 


579 


55.9 


43.0 


No 


456 


44.1 


57.0 



In other words, 56 percent of the NELS:88 school sample, but 43 percent of the population of 
38,774 schools with eighth grades, have academic honor societies available to eighth graders. 

Weighted estimates from NELS:88 should approximate estimates from other national sources, 
within the combined sampling error of the two sources. (We say "approximate" because 
sampling variance is not the only possible source of measurement error; estimates can be 
biased by nonresponse, or by errors in reporting, data collection, or data processing). Use of 
external sources (and the NELS:88 codebook as well) as a benchmark may thus provide a 
check on whether the analyst is using data correctly. 

For several reasons, weighted estimates for rare populations may lack precision. For example, 
NELS:88 data permits the analyst to examine the educational situation of Japanese-Americans 
attending suburban schools in the southern region. Because of small sample sizes, sampling 
error will be comparatively large. Moreover, while nonresponse adjustments in the weights 
compensate for broad categories of nonrespondents, including nonresponding Asians, it is not 
possible to adjust the weights for all factors that may be associated with nonresponse. 
Differential response between Asian subgroups, between urban and suburban Asians, or 
between Asians in different regions, is not taken into account in the generation of final 
weights, and may bias estimates for certain subgroups. Hence rare populations in NELS:88 
are not guaranteed to produce precise population estimates. 

Standard Errors. Standard errors are a measure of sampling variability in survey results; the 
standard error is an estimate of the expected difference between a statistic from a particular 
sample and the corresponding population value. The standard error of measurement is critical 
to determining the statistical significance of statistical inferences and interpretations. Because 
the NELS:88 sample design involved stratification, disproportionate sampling of certain strata, 
and clustered (i.e., multi-stage) probability sampling, the resulting statistics are more variable 
than they would have been had they been based on data from a simple random sample of the 
same size. 

Popular statistical analysis packages such as SPSS or SAS do not calculate standard errors 
that take into account complex sample designs. Standard errors that do take into account a 
complex design such as that of NELS:88 will generally be higher than standard errors 
calculated under the assumptions of simple random sampling. Several procedures are available 
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for estimating the variance or sampling errors for complex samples. Procedures such as Taylor 
series approximations, Balanced Repeated Replication (BRR), and Jackknife Repeated 
Replication (JRR) produce similar results. Given the structure of the NELS.88 database, users 
are likely to find the Taylor series procedure the most convenient means to calculate standard 
errors. Software available for producing Taylor Series standard errors include C-Tab 
(available from NCES), SUDAAN, and SUPER CARP, among other variance estimation 
programs. 

Alternatively, the design effect can be used to quantify the impact of departures from simple 
random sampling on the precision of sample estimates, and can serve as an adjustment factor 
for variances or standard errors calculated in accordance with the assumption of a simple 
random sample. For any statistical estimator (for example, a mean or a proportion), the design 
effect (DEFF) is the ratio of the estimate of the variance of a statistic derived from 
consideration of the sample design to that obtained from the formula for simple random 
samples. Because in many cases it is more useful to work with standard errors than with 
variances, the square root of the design effect (also called the root design effect, and 
designated as DEFT) is also useful. The root design effect expresses the relation between the 
actual standard error of an estimate and the standard error of the corresponding estimate from 
a simple random sample. A detailed account of design effects is provided in the NELS:88 
user manuals and the sample design and final technical reports. 

Selecting the Correct Weight. Owing to the complexity of the NELS:88 sample design, 
weighted data must be used to ensure accurate estimation and inference. In many cases, 
weights must be used with a particular flag (together with weight) in order to analyze the 
sample for a particular target population. The following nonresponse-adjusted weights appear 
on the data files: 

Base Year 



BYQWT. Use this weight for cross-sectional analysis of 1988 eighth graders when 
student is the unit of analysis. 

BYADMWT. Use this weight for cross-sectional analysis of 1988 schools attended by 
eighth graders when school is the unit of analysis. 

First Followup 

F1QWT. Use this weight for cross-sectional analysis of NELS:88 sample members in 
1990 when interested in making inferences about the population of tenth graders. 

F1PNLWT. Use this weight to analyze 1988 and 1990 data in the examination of 
1988 eighth graders two years later. 

F1DQAJWT. Use this weight when analyzing dropout questionnaire items that were 
not included in the abbreviated version of the dropout questionnaire. (See the 1990 
dropout user manual for details on use of F1DQAJWT and F1DPAJWT). 
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F1DPAJWT. This is the 1988-90 panel version of F1DQAJWT, suitable for use when 
both base-year and first followup data will be used in analysis. 

Second Followup 

F2QWT. Use this weight for cross-sectional analysis of NELS:88 sample members in 
1992 when interested in making inferences about the population of twelfth graders. 

F2F1PNWT. Use this weight for producing weighted student panel statistics when 
both first followup and second followup data are employed in analysis (for example, to 
measure changes in the 1990 sophomore cohort two years later). 

F2PNLWT. Use for producing weighted student panel statistics when all three waves 
(survey base-year through second followup) are included in the analysis. 

F2CXTWT. Use this weight for analyzing 1992 teacher and school administrator data. 
This cross-sectional weight applies to students who attended the schools selected for 
inclusion in the teacher and school administrator components and who completed a 
second followup questionnaire. The population was restricted to early graduates and 
students who were in the schools during spring data collection. This weight allows 
analysts to generate national statistics at student level using the teacher and school 
administrator data despite the bias against small cluster sizes in sample selection. 

F2TRP1WT. This panel weight applies to sample members who were participants in 
1988, 1990, and 1992 (all three rounds of NELS:88) and for whom transcript data are 
available. F2TRP1WT allows analysts to perform panel analyses using transcript data 
in conjunction with 1988, 1990, and 1992 test and questionnaire data. 

F2TRP2WT. This panel weight applies to sample members who were participants in 
1990 and 1992 (the first and second followups) and for whom transcript data are 
available. F2TRP2WT allows analysts to perform panel analyses using transcript data 
in conjunction with 1990-1992 test and questionnaire data. 

F2TRSCWT. Use this weight for analysis involving high school transcript data. This 
cross-sectional weight applies to all early graduates, dropouts, students in sampled 
schools during spring data collection, and all sample members who were both 
ineligible for all three rounds of NELS:88 and were in the twelfth grade during the 
1991-92 school year for whom a high school transcript was received. 

F2PAQWT. Use this weight when analyzing 1992 parent data. This cross-sectional 
weight applies to all students (student level weight) for whom a parent questionnaire 
was collected during the second followup. 

Using Weights. Table 7. 1.1-1 of the first followup and second followup user’s manual 
codebooks defines analysis samples and populations, with appropriate weights and flags. For 
example, for the first followup student data, four populations of interest are identified, the 
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number of sample members for each is specified, and weights and flags indicated, the first of 
which is the 1988-1990 panel — 1988 eighth graders two years later. To analyze this 
population, the table indicates that the flag F1PANFLG should be selected, and the weight 
F1PNLWT employed. 

Weights have not been constructed for all possible analytic purposes. In instances where there 
is a population of interest for which no specific weight has been produced, some existing 
weights may provide reasonable approximations. It was for this reason that no separate parent 
weight was produced in the NELS:88 base-year, and that student weights adjusted for 
cognitive test nonresponse have not been generated. Base-year parent — and test 
completion — response rates were so high relative to student questionnaire completion, that the 
student weight can be used for these cases with only minimal bias. 

The second followup contextual (school) and parent weights exist only in a cross-sectional 
version. This poses a difficulty (in selecting a weight) if in longitudinal analyses, for example, 
you wanted to look at parent-reported changes in the home education support system between 
1988 and 1992. One might employ the 1992 cross-sectional parent weights in conjunction 
with prior round data. However, since these weights were not produced for this purpose, 
population estimates produced from them would contain biases. 

Extreme caution must be exercised when a researcher wishes to conduct a weighted analysis 
for which a specific weight has not been provided. In particular, the analyst must weigh how 
much bias will be introduced by substitution of another weight. Two checks that should be 
made are (1) examination of the sum of the weights to compare that sum to true population 
totals; (2) analysis of nonresponse bias, that is of what groups are disproportionately excluded 
by the weighting strategy. The specific purpose and required degree of precision of the 
analysis must be taken into account. If subgroup estimates are central to the analytic plan, it is 
necessary to take into account bias at the subgroup level. In instances where more them one 
possible weight is available, the decision as to which is best to use should be guided by a 
comparison of the bias inherent in each alternative. Whatever weighting strategy is used, if the 
amount of bias is non-trivial, adjustments may be incorporated in models that accommodate 
measurement error, and potential biases reported with the results. 

REFERENCES 

Base-year sample design report ; chapter 3 of the base-year, the first followup, and the second 
followup student component user manuals; the forthcoming second followup sample design 
report. 

Working with Missing Data 

In working with the NELS:88 data, it is important to understand the sources of missing data. 
The two main types of missing data in NELS:88 are "unit" and "item" nonresponse (i.e., 

"unit" signifies nonresponse to the entire survey instrument; "item" signifies nonresponse to 
specific items on the survey instrument). For example, 1,446 sampled students and/or dropouts 
did not complete first followup survey instruments (unit nonresponse). For item F1S78B 
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(number of times respondent drank alcohol during last 12 months), an additional 2,425 
students left the item blank (item nonresponse). 

In the NELS:88 first followup student codebook, four codes are used to signify values that the 
researcher may want to declare missing for specific analyses. These are: 

1 . Nonrespondents & Dropouts (blank) - the respondent is either a nonrespondent or a 
dropout (for item F1S78B on student questionnaire, the number of nonrespondents and 
dropouts equaled 2,485). 

2. Missing (8, 98, 998) - the respondent did not respond to a particular item or group of 
items (for item F1S93E, the number of missing equaled 1,495). 

3. Refused (7, 97, 997) - the respondent refused to respond to a particular item or group 
of items (for item F1S93E, the number of refused equaled 36). 

4. Legitimate skip (9, 99, 999) - a respondent’s response to a previous filter question 
allows that respondent to skip a particular item or group of items (for item F1S78B, 
the number of legitimate skips equaled 366). 

It is important for the analyst to realize that missing data has the potential to BIAS analyses. 
Nonrespondents may have different characteristics than respondents (e.g., a higher proportion 
of nonrespondents may be classified as low SES). It is important to compare the 
characteristics of the respondents in the working data file with the characteristics of both the 
total population and the nonrespondents who will be deleted from the analysis because of 
missing data. 

REFERENCES 

Second Followup Student User’s Manual (section III-3.4; section VII-7.3) 

Choosing between Cross-sectional and Longitudinal Analyses 

NELS:88 data can be used descriptively, or inferentially, in multivariate models, and both 
cross-sectional and longitudinal analyses can be performed on the data. 

NELS:88 data can be used in two distinct ways. One use is descriptive and aims at 
estimation — specifically, the estimation of descriptive and relational population and subgroup 
statistics. Such descriptive data are usually expressed as means (for example, mean gain in 
reading scores for various racial/ethnic groups between 1988 and 1990) or percentages (for 
example, the percentages of male and female 1988 eighth graders who report studying more 
in high school than in eighth grade) and answer questions about the size and distribution of 
educational phenomena or about the characteristics of students and the educational system. For 
example, descriptive data tell us how many students have dropped out of school, and who 
they are — that is, whether they are more likely to be male or female, or how much more 
likely high achieving students are to stay in school. But descriptive accounts, while they 
answer the question "how many", cannot conclusively answer the question "why". A second 
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way of using the data, however, can address such questions. "Why" questions involve using ' 
the data inferentially, aiming at explanatory accounts of educational processes. Inferential uses 
can assume either a research or an applied policy focus. Research investigations into 
educational processes and outcomes test hypotheses about the underlying causes of educational 
phenomena (i.e., What school related factors are associated with school failure?). 

In a longitudinal design, a probability sample of a population is drawn at one time (for 
NELS:88, 1988 eighth graders) and the same individuals are measured at later times (for 
NELS:88, in 1990, 1992, 1994, and 1998). However, in a cross-sectional survey a different 
probability sample is drawn of the population at each point of time when the survey is 
conducted. The base year of a longitudinal survey is also, by definition, a cross-sectional 
survey. NELS:88 base-year data can be analyzed cross-sectionally — researchers can look at 
the situation of a nationally representative sample of eighth graders in 1987-88. Or NELS:88 
base-year data can be analyzed longitudinally — one can look at what happens to the eighth 
grade cohort over time (for example, one can measure the gain in their mathematics 
achievement between 1988 and 1992, or who drops out of school between 1988 and 1990). 

NELS:88 has a special sample freshening feature which effectively supplies two additional 
nationally representative cross-sections: the nation’s spring 1990 sophomores and spring 1992 
seniors. Hence the first and second followup data can also be analyzed either cross-sectionally 
or longitudinally. 

Cross-sectional analysis provides a snapshot at a single point in time. Repeated cross-sectional 
analysis, and longitudinal analysis, permit the measurement of change over time. In terms of 
the sample design, change (and stability over time) can be measured in one of two ways in 
NELS:88: 

1 . At the group level, change can be measured across the successive cross- 
sections — eighth graders in 1988, sophomores in 1990, and seniors in 1992. In 
addition, NELS:88 and comparable studies (e.g., NLS-72 and HS&B) can be analyzed 
as repeated cross-sections (e.g. of seniors in 1972, 1980/82, and 1992) to measure 
trends. 

2. Change can also be analyzed in individuals over time. The latter possibility — true 
longitudinal measurement — represents, for most purposes, the unique strength of the 
NELS:88 design. Following individual educational histories generally provides the best 
basis for drawing causal inferences about educational processes and their effects. 

To analyze followup data cross-sectionally, one must carefully define the analysis sample. For 
example, the eighth grade cohort in 1990 includes tenth grade students, students not in tenth 
grade, and dropouts, but does not include freshened tenth graders. On the other hand the 
sophomore cohort in 1990 includes freshened students and all members of the eighth grade 
cohort who were enrolled in the tenth grade in the spring of 1990. With release of data from 
the second followup, three cohorts can be analyzed with the help of appropriate weights and 
flags: two cross-sectionally or longitudinally (eighth grade and sophomore) and one cross- 
sectionally only (1992 seniors). 
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Special Procedures for Regression and Other Multivariate Techniques 

NCES longitudinal surveys are based on complex samples that use stratified cluster designs. 
This has an impact on the estimates of variances (i.e., standard errors) for the statistical 
analyses carried out on surveys such as NELS:88. Standard statistical packages such as SAS 
and SPSS treat data as if they were collected under a simple random sample design. There are 
two important aspects of the NELS:88 survey which make it different from a simple random 
sample design: (1) the unequal probabilities of selection (e.g., NELS:88 oversampled 
minorities, giving minorities a higher probability of selection than nonminorities) and (2) the 
clustered nature of NELS:88 designs (e.g., NELS:88 sampled schools (clusters) and then 
students rather than simply taking a sample of students directly). The impact of not having a 
simple random sample is that the observations are not independent and the variance of 
an estimate is not calculated using the standard textbook formula for s 2 . A sound 
approach to the analysis will account for these two characteristics of the NELS:88 sample. 

Recommended Statistical Procedures. The following are recommended procedures for 
carrying out regression and other multivariate techniques on data from complex sample 
surveys such as NELS:88. 

1. Preferred Procedure: Use a program designed specifically for analyzing data 
from complex surveys. 

SUDAAN, a program which uses a Taylor series approach and is available for the mainframe 
or the PC is the most widely used. Others to be considered are PCCARP (Taylor series 
approach, PC), and CPLX (Bob Faye’s (Census Bureau) program for contingency table 
analysis) or VPLX (currently able to compute variances for means and proportions and being 
expanded by Faye to do regression and other analyses). An NCES contractor has successfully 
used PCCARP and SUDAAN in recent NCES reports for regression and logistic regression 
analysis. 

2 . Alternative for Ease of Implementation: Use SAS or SPSS or other statistical 
package using relative weights as described in 3 below, with or without design 
effect adjustment (as described in 3 below) for exploratory work. When down to 
a few plausible models, run SUDAAN or another package designed for survey 
data (as in 1 above). 

This approach saves time by doing preliminary runs with a faster, easier to use package, but 
checks out final results building in the survey design. 

3 . Third in Preference but Acceptable Procedure: Use SAS or SPSS or other 
statistical package with relative weights and a design effect correction to the 
standard error. 

This approach gives an approximate and probably conservative adjustment; i.e., there may be 
real differences which are not picked up. It is implemented as follows: 
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(a) Run the analysis using a standard statistical package which allows the use of weights. 
The weight to be used in the analysis is the relative weight: the final survey weight 
divided by the average weight for the group being analyzed: 

relwti = wti/avg wt. 

Using the survey weight in the analysis accounts for the oversampling of certain populations 
and redistributes the observations to represent the distribution in the population. Dividing by 
the average weight keeps the correct distribution but makes the weights add to n, the sample 
size for your analysis. Failure to do this would lead to huge numbers of observations, which 
would inflate the degrees of freedom and the significance of the test statistic. 

(b) Adjust the standard error to account for the complex nature of the design. To do this, 
multiply the standard error by the square root of the design effect for the dependent 
variable for the total group being analyzed. For example, if we are regressing test 
score on race/ethnicity represented by four dummy variables, this procedure would 
multiply the simple random sample standard error by the square root of the design 
effect for test score calculated for the total group being analyzed. Such design effects 
are usually found in the datafile user’s manual. If the design effect for your dependent 
variable is not available, you might use the DEFT for a similar variable or the average 
design effect averaged over a set of variables if that is all that is available. 

It is also possible to use the average design effect of the dependent variable, averaged over 
subgroups represented by the independent variables in the model. In the above example, four 
design effects would be calculated: the design effect of the mean test score, calculated for 
each of the four race/ethnicity subgroups. The square root of the average of these four design 
effects would be used to adjust the standard error. Equivalently, rather than multiplying the 
standard error by the square root of the design effect, one could divide the t statistic by the 
square root of the design effect or divide the F statistic by the design effect. 

(c) The correct degrees of freedom for the error term in these analyses is the same as that 
given by SAS; i.e., the simple random sample degrees of freedom: sample size - 
number of terms in model. 
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Must Read Publications 

Before a researcher attempts to use the NELS:88 data files, it is strongly suggested that time 
be spent reading the NELS:88 user’s manuals and design documents. The following list of 
documents will provide researchers with the essential information that they need to understand 
the complexities of the NELS:88 data files. 

Base-year Sample Design Report - This report documents the design and conduct of 
data collection activities for the NELS:88 base-year survey of eighth graders. Much of 
the information contained in this report is also contained in the base-year user’s 
manuals. An updated Sample Design report will be produced for the second followup. 

Base-year User’s Manuals - Four user’s manuals cover the student, school, teacher, 
and parent components of the base-year surveys administered in 1988. 

First Followup User’s Manuals - Four user’s manuals cover the student, dropout, 
school, and teacher components of the first followup surveys administered in 1990. 

Second Followup Public Use User’s Manuals - Five user’s manuals cover the 
student, dropout, school, teacher, and parent components of the second followup 
surveys administered in 1992. 

Second Followup Privileged Use User’s Manuals - The Transcript User’s Manual 
describes the activities involved in collecting high school transcript data from 
NELS:88 cohort members. A user’s manual will also be produced for the 1990-92 
School Effectiveness Study. 

Psychometric documentation is available in the Base-year Psychometric Report and the 
(forthcoming) Second Followup Psychometric Report. 

Owings, J. et al. (1994) A Guide to using NELS:88 data gives user-friendly technical guide 
to using NELS:88 data. 
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NCES and Contractor Staff of Technical Expertise 

Staff at NCES, NORC, ETS, and MPR Associates have been closely associated with 
important components of the NELS:88 design, instrumentation, sampling, data collection, data 
processing, and analyses that have occurred over the life of this project. These individuals, 
their specific expertise, and their phone numbers are included in the following list. 



Overall Knowledge of NELS:88 

Jeff Owings (NCES) 

Jerry West (NCES) 

Marilyn McMillen (NCES) 

Steven Ingels (NORC) 

Leslie Scott (NORC) 

Sampling 

Ralph Lee (NCES) 

Steven Ingels (NORC) 

Martin Frankel (NORC) 

Instrumentation 

Peggy Quinn (NCES) 

Leslie Scott (NORC) 

Test Development 

Don Rock (ETS) 

Analyses 

Jeff Owings (NCES) 

Jerry West (NCES) 

Marilyn McMillen (NCES) 

Steven Ingels (NORC) 

Leslie Scott (NORC) 

Phil Kaufman (MPR) 



( 202 ) 219-1777 
( 202 ) 219-1574 
( 202 ) 219-1754 

( 312 ) 753-7565 
( 312 ) 753-7564 



( 202 ) 219-1732 

( 312 ) 753-7565 

( 203 ) 869-1324 



( 202 ) 219-1743 
( 312 ) 753-7564 



( 609 ) 734-5655 



( 202 ) 219-1777 
( 202 ) 219-1574 
( 202 ) 219-1754 

( 312 ) 753-7565 
( 312 ) 753-7564 

( 510 ) 849-4942 
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Baccalaureate and Beyond Study (B&B) - A national longitudinal survey that provides 
information on education and work experiences after completing the bachelor’s degree. 

Beginning Postsecondary Student Longitudinal Study (BPS) - A national survey on 
participation, persistence, and attainment in postsecondary education, representing the national 
population of postsecondary students (including "nontraditional" college goers as well as 
traditional high school graduates). 

Common Core of Data (CCD) - The NCES comprehensive data source on the nation’s 
elementary and secondary schools, containing information on school identification, 
administrative and financing data. 

Early Childhood Longitudinal Study (ECLS) - An upcoming NCES project that will 
survey children of preschool age with followups through their young adulthood. 

High School and Beyond (HS&B) - The second in the series of longitudinal education 
studies sponsored by NCES. The HS&B Base Year study surveyed sophomore and senior 
students in 1980, with four followups through 1992. 

Integrated Postsecondary Education Data System (IPEDS) - The NCES single, 
comprehensive data system including all identified postsecondary education institutions. 

National Assessment of Educational Progress (NAEP) - A congressionally mandated 
project to monitor continuously the knowledge, skills, and performance of the nation’s 
children and youth. It provides data about student performance at national, regional, and, on a 
trial basis, state levels. 

National Household Education Survey (NHES) - This is the first NCES survey to collect 
education data through a household-based survey. Each collection covers two topical 
components. 

National Longitudinal Study of the High School Class of 1972 (NLS-72) - This project 
was the first in the series of longitudinal education studies sponsored by NCES. It surveyed a 
cohort of 1972 senior students with four followups through 1986. 

National Postsecondary Student Aid Study (NPSAS) - A nationwide study of students 
enrolled in less-than-2-year institutions, community and junior colleges, 4-year colleges and 
major universities in the U.S. and Puerto Rico. Students who receive financial aid, as well as 
those who do not receive aid, participate in the survey. 

National Survey of Postsecondary Faculty (NSOPF) - A national survey that provides data 
about faculty to postsecondary education researchers, planners, and policymakers. NSOPF is 
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the most comprehensive study of faculty in postsecondary educational institutions ever 
undertaken. 

Recent College Graduates Study (RCG) - A survey of college graduates conducted 
periodically from 1976 to 1991, providing information about post-degree employment and 
education experiences among this population, with estimates on potential teacher supply. Since 
1993, the Baccalaureate and Beyond Study (B&B) has replaced RCG. 



Schools and Staffing Survey (SASS) - A national survey of schools, local education 
agencies, school administrators, and teachers; one component is the Teacher Followup Survey 
(TFS). 

Teacher Followup Survey (TFS) - See Schools and Staffing Survey. 
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94-01 

94-02 

94-03 

94-04 

94-05 

94-06 

94- 07 

95- 01 
95-02 
95-03 



Listing of NCES Working Papers to Date 



Title 



Schools and Staffing Survey (SASS) 
Papers Presented at Meetings of the 
American Statistical Association 

Generalized Variance Estimate for 
Schools and Staffing Survey (SASS) 

1991 Schools and Staffing Survey (SASS) 
Reinterview Response Variance Report 

The Accuracy of Teachers’ Self-reports 
on their Postsecondary Education: Teacher 
Transcript Study, Schools and Staffing 
Survey 

Cost-of-Education Differentials 
Across the States 

Six Papers on Teachers from the 
1990-91 SASS and Other Related 
Surveys 

Data Comparability and Public Policy: 
New Interest in Public Library Data 
Papers Presented at Meetings of the 
American Statistical Association 

Schools and Staffing Survey: 1994 
papers presented at the 1994 Meeting 
of the American Statistical 
Association 

QED Estimates of the 1990-91 Schools 
and Staffing Survey: Deriving and 
Comparing QED School Estimates with 
CCD Estimates 

Schools and Staffing Survey: 1990-91 
SASS Cross-Questionnaire Analysis 



Contact 



Dan Kasprzyk 
Dan Kasprzyk 
Dan Kasprzyk 



Dan Kasprzyk 
William Fowler 



Dan Kasprzyk 



Carrol Kindel 



Dan Kasprzyk 



Dan Kasprzyk 
Dan Kasprzyk 



Listing of NCES Working Papers to Date (Continued) 



Number 



Title 



Contact 



95-04 

95-05 

95-06 

95-07 

95-08 

95-09 

95-10 

95-11 

95-12 



National Education Longitudinal 
Study of 1988: Second Follow-up 
Questionnaire Content Areas and 
Research Issues 

National Education Longitudinal 
Study of 1988: Conducting Trend 
Analyses of NLS-72, HS&B, and 
NELS:88 Seniors 

National Education Longitudinal 
Study of 1988: Conducting Cross-Cohort 
Comparisons Using HS&B, NAEP, and 
NELS:88 Academic Transcript Data 

National Education Longitudinal 

Study of 1988: Conducting Trend Analyses 

HS&B and NELS:88 Sophomore Cohort 

Dropouts 

CCD Adjustments to the 1990-91 SASS: 

A Comparison of Estimates 

The Results of the 1993 Teacher List 
Validation Study (TLVS) 

The Results of the 1991-92 Teacher 
Follow-up Survey (TFS) Reinterview and 
Extensive Reconciliation 

Measuring Instruction, Curriculum Content, 
and Instructional Resources: The Status 
of Recent Work 

Rural Education Data 
User’s Guide 



Jeffrey Owings 



Jeffrey Owings 



Jeffrey Owings 



Jeffrey Owings 



Dan Kasprzyk 



Dan Kasprzyk 



Dan Kasprzyk 



Sharon Bobbitt & 
John Ralph 
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Number 



Title 



Contact 



95-13 

95-14 

95-15 

95-16 

95-17 

95- 18 

96- 01 
96-02 

96-03 



Assessing Students with Disabilities 
and Limited English Proficiency 

Empirical Evaluation of Social, 
Psychological, & Educational 
Construct Variables Used in 
NCES Surveys 

Classroom Instructional Processes: 

A Review of Existing Measurement 
Approaches and Their Applicability 
for the Teacher Follow-up Survey 

Intersurvey Consistency in NCES 
Private School Surveys 

Estimates of Expenditures for 
Private K-12 Schools 

An Agenda for Research on 
Teachers and Schools: 

Revisiting NCES’ Schools and 
Staffing Survey 

Methodological Issues in the Study 
of Teachers’ Careers: Critical Features 
of a Truly Longitudinal Study 

Schools and Staffing Survey (SASS): 1995 
Selected papers presented at the 1995 
Meeting of the American Statistical 
Association 

National Education Longitudinal Study of 
1988 (NELS:88) Research Framework and 
Issues 



James Houser 



Samuel Peng 



Sharon Bobbitt 



Steven Kaufman 



Steve Broughman 



Dan Kasprzyk 



Dan Kasprzyk 



Dan Kasprzyk 



Jeffrey Owings 
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